[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:29 dist-test-slave-qg60 krb5kdc[9387](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:29 dist-test-slave-qg60 krb5kdc[9387](info): set up 2 sockets
Aug 09 02:14:29 dist-test-slave-qg60 krb5kdc[9387](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260809 02:14:30.890334 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:30 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:31.025211 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38447
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38447
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:31.133025 9402 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:31.133258 9402 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:31.133282 9402 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:31.134392 9402 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:31.134430 9402 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:31.134445 9402 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:31.134457 9402 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:31.134469 9402 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:31.135960 9402 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38447
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38447
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9402
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:31.136183 9402 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:31.136370 9402 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:31.141526 9408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.141597 9409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.141775 9411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:31.141780 9402 server_base.cc:1034] running on GCE node
I20260809 02:14:31.142174 9402 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:31.142493 9402 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:31.143741 9402 hybrid_clock.cc:648] HybridClock initialized: now 1786241671143741 us; error 103 us; skew 500 ppm
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:31.146615 9402 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:31.147210 9402 webserver.cc:460] Webserver started at http://127.9.40.126:40149/ using document root <none> and password file <none>
I20260809 02:14:31.147497 9402 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:31.147568 9402 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:31.147678 9402 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:31.148924 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "584dd29b268346c0bc04a84132a5b371"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "3bf567e3c4db1af175b411cd2f952a0a"
server_key_iv: "4c4b368b4639cf8653df908b30373ce5"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.149253 9402 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "584dd29b268346c0bc04a84132a5b371"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "3bf567e3c4db1af175b411cd2f952a0a"
server_key_iv: "4c4b368b4639cf8653df908b30373ce5"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.150557 9402 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:14:31.151383 9418 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.151575 9402 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:31.151687 9402 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "584dd29b268346c0bc04a84132a5b371"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "3bf567e3c4db1af175b411cd2f952a0a"
server_key_iv: "4c4b368b4639cf8653df908b30373ce5"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.151767 9402 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:31.165489 9402 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:31.168073 9402 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:31.168260 9402 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:31.172605 9402 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38447
I20260809 02:14:31.172648 9475 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38447 every 8 connection(s)
I20260809 02:14:31.173051 9402 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:31.173981 9477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:31.176358 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371: Bootstrap starting.
I20260809 02:14:31.177016 9477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:31.177294 9477 log.cc:824] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:31.178030 9477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371: No bootstrap required, opened a new log
I20260809 02:14:31.179603 9477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } }
I20260809 02:14:31.179769 9477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:31.179793 9477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 584dd29b268346c0bc04a84132a5b371, State: Initialized, Role: FOLLOWER
I20260809 02:14:31.179940 9477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } }
I20260809 02:14:31.180025 9477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:31.180049 9477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:31.180079 9477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:31.180650 9477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } }
I20260809 02:14:31.180932 9477 leader_election.cc:302] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 584dd29b268346c0bc04a84132a5b371; no voters:
I20260809 02:14:31.181074 9477 leader_election.cc:288] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:31.181136 9480 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:31.181331 9480 raft_consensus.cc:686] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [term 1 LEADER]: Becoming Leader. State: Replica: 584dd29b268346c0bc04a84132a5b371, State: Running, Role: LEADER
I20260809 02:14:31.181365 9477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:31.181432 9480 consensus_queue.cc:237] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } }
I20260809 02:14:31.181908 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 9402
I20260809 02:14:31.182030 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:31.182209 9483 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 584dd29b268346c0bc04a84132a5b371. Latest consensus state: current_term: 1 leader_uuid: "584dd29b268346c0bc04a84132a5b371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } } }
I20260809 02:14:31.182353 9483 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:31.182405 9481 sys_catalog.cc:455] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "584dd29b268346c0bc04a84132a5b371" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "584dd29b268346c0bc04a84132a5b371" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38447 } } }
I20260809 02:14:31.182502 9481 sys_catalog.cc:458] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:31.182726 9490 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:31.182880 9377 external_mini_cluster.cc:1428] Setting key 11df4dc9eef130db5f9e3be705bf0020
I20260809 02:14:31.183382 9490 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:31.185174 9490 catalog_manager.cc:1347] Generated new cluster ID: f3d3ff658b3b4be4b70e44742abeed6c
I20260809 02:14:31.185236 9490 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:31.192312 9490 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:31.192976 9490 catalog_manager.cc:1504] Loading token signing keys...
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241670, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:31.200803 9490 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 584dd29b268346c0bc04a84132a5b371: Generated new TSK 0
I20260809 02:14:31.201028 9490 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:31.202544 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.183851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59350 (local address 127.9.40.126:38447)
0809 02:14:31.184865 (+ 1014us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.184874 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.184916 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:14:31.186798 (+ 1882us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.186806 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.186865 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.187020 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.189038 (+ 2018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.191075 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.192527 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.193218 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.193335 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.197939 (+ 4604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.197944 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.197944 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.197959 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.200087 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.200750 (+ 663us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.200755 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.200755 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.200891 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.201217 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.201219 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.201219 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.201494 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.201639 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.202088 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.202261 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":963,"thread_start_us":100,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:31.232384 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:31.345001 9505 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:31.345273 9505 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:31.345321 9505 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:31.346645 9505 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:31.346719 9505 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:31.346767 9505 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:31.348594 9505 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9505
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:31.348920 9505 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:31.349177 9505 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:31.355892 9511 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.356050 9514 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.356156 9512 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:31.356433 9505 server_base.cc:1034] running on GCE node
I20260809 02:14:31.356699 9505 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:31.357038 9505 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:31.358243 9505 hybrid_clock.cc:648] HybridClock initialized: now 1786241671358231 us; error 46 us; skew 500 ppm
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:31.361377 9505 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:31.361904 9505 webserver.cc:460] Webserver started at http://127.9.40.65:37137/ using document root <none> and password file <none>
I20260809 02:14:31.362159 9505 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:31.362231 9505 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:31.362382 9505 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:31.363595 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "29685cf36280440faa0c23d78ea13585"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "535d5a5f78b7aa1a8ced4d787445bb26"
server_key_iv: "1d791198b9a6d2c29889845ce803a9a7"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.364013 9505 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "29685cf36280440faa0c23d78ea13585"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "535d5a5f78b7aa1a8ced4d787445bb26"
server_key_iv: "1d791198b9a6d2c29889845ce803a9a7"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.365835 9505 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:14:31.366818 9521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.367071 9505 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:31.367198 9505 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "29685cf36280440faa0c23d78ea13585"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "535d5a5f78b7aa1a8ced4d787445bb26"
server_key_iv: "1d791198b9a6d2c29889845ce803a9a7"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.367278 9505 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:31.384101 9505 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:31.386525 9505 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:31.386749 9505 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:31.387154 9505 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:31.387480 9505 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:31.387535 9505 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.387580 9505 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:31.387605 9505 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.402871 9505 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45269
I20260809 02:14:31.403095 9644 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45269 every 8 connection(s)
I20260809 02:14:31.403345 9505 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:31.409389 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 9505
I20260809 02:14:31.409505 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:31.409660 9377 external_mini_cluster.cc:1428] Setting key 79777075529d8030a6c767525e6f910c
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:31.416093 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.404300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35201 (local address 127.9.40.126:38447)
0809 02:14:31.404389 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.404406 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.404773 (+ 367us) server_negotiation.cc:407] Connection header received
0809 02:14:31.405616 (+ 843us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.405617 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.405627 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.405684 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.406854 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.407388 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.408574 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.408804 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.408862 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.411873 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.411875 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.411875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.411883 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.414516 (+ 2633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.415066 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.415067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.415067 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.415121 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.415428 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.415428 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.415429 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.415532 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.415582 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.415945 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.416026 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:14:31.416471 9648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.404267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38447 (local address 127.9.40.65:35201)
0809 02:14:31.404610 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:31.404649 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:31.405516 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:31.405752 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:31.405755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:31.406472 (+ 717us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:31.406751 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.406756 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.407490 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.407491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.408484 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.408487 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.408885 (+ 398us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.408886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.409075 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.409887 (+ 812us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:31.409899 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:31.411778 (+ 1879us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:31.414608 (+ 2830us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.414622 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.414624 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.414988 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.415198 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.415199 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.415199 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.415359 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.415598 (+ 239us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:31.415605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:31.415796 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 02:14:31.415926 (+ 130us) client_negotiation.cc:240] Negotiation successful
0809 02:14:31.416118 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":218,"threads_started":1}
I20260809 02:14:31.417081 9645 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38447
I20260809 02:14:31.417169 9645 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:31.417390 9645 heartbeater.cc:511] Master 127.9.40.126:38447 requested a full tablet report, sending...
I20260809 02:14:31.417938 9436 ts_manager.cc:194] Registered new tserver with Master: 29685cf36280440faa0c23d78ea13585 (127.9.40.65:45269)
I20260809 02:14:31.418594 9436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:35201
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:31.443663 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:31.550771 9651 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:31.551112 9651 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:31.551146 9651 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:31.552330 9651 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:31.552374 9651 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:31.552402 9651 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:31.554049 9651 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9651
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:31.554296 9651 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:31.554530 9651 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:31.559752 9656 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.559813 9657 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.559801 9659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:31.559955 9651 server_base.cc:1034] running on GCE node
I20260809 02:14:31.560298 9651 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:31.560551 9651 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:31.561695 9651 hybrid_clock.cc:648] HybridClock initialized: now 1786241671561672 us; error 38 us; skew 500 ppm
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:31.564178 9651 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:31.564647 9651 webserver.cc:460] Webserver started at http://127.9.40.66:43143/ using document root <none> and password file <none>
I20260809 02:14:31.564927 9651 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:31.564975 9651 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:31.565122 9651 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:31.566169 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "330d8e1e8e964afc8f2c3a3d00865ad3"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "9a28f69f6b46f6fdd12c46b2ee4e4fc4"
server_key_iv: "2006bf3323b71e94976002615cf7776b"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.566526 9651 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "330d8e1e8e964afc8f2c3a3d00865ad3"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "9a28f69f6b46f6fdd12c46b2ee4e4fc4"
server_key_iv: "2006bf3323b71e94976002615cf7776b"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.568074 9651 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:31.569021 9666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.569204 9651 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:31.569314 9651 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "330d8e1e8e964afc8f2c3a3d00865ad3"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "9a28f69f6b46f6fdd12c46b2ee4e4fc4"
server_key_iv: "2006bf3323b71e94976002615cf7776b"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.569389 9651 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:31.583853 9651 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:31.586279 9651 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:31.586498 9651 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:31.586920 9651 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:31.587325 9651 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:31.587386 9651 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.587422 9651 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:31.587467 9651 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.594781 9651 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:44701
I20260809 02:14:31.594826 9790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:44701 every 8 connection(s)
I20260809 02:14:31.595201 9651 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:31.600149 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 9651
I20260809 02:14:31.600250 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:31.600405 9377 external_mini_cluster.cc:1428] Setting key b002dcb5416cdcd7fb066c98c46465ee
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:31.606621 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.596000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50053 (local address 127.9.40.126:38447)
0809 02:14:31.596085 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.596088 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.596385 (+ 297us) server_negotiation.cc:407] Connection header received
0809 02:14:31.596989 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.596990 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.596999 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.597054 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.598261 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.599362 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.600759 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.600996 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.601079 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.603312 (+ 2233us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.603314 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.603314 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.603322 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.605127 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.605531 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.605532 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.605533 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.605599 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.605912 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.605913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.605913 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.606072 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.606148 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.606410 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.606501 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:14:31.606657 9794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.595968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38447 (local address 127.9.40.66:50053)
0809 02:14:31.596255 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:31.596293 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:31.596892 (+ 599us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:31.597175 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:31.597179 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:31.597819 (+ 640us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:31.598141 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.598146 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.599499 (+ 1353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.599500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.600636 (+ 1136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.600639 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.601088 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.601089 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.601228 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.601825 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:31.601834 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:31.603212 (+ 1378us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:31.605222 (+ 2010us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.605229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.605230 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.605450 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.605679 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.605679 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.605680 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.605836 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.606145 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:31.606150 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:31.606274 (+ 124us) client_negotiation.cc:769] Sending connection context
0809 02:14:31.606360 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:14:31.606489 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":198,"threads_started":1}
I20260809 02:14:31.607299 9791 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38447
I20260809 02:14:31.607393 9791 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:31.607577 9791 heartbeater.cc:511] Master 127.9.40.126:38447 requested a full tablet report, sending...
I20260809 02:14:31.607956 9436 ts_manager.cc:194] Registered new tserver with Master: 330d8e1e8e964afc8f2c3a3d00865ad3 (127.9.40.66:44701)
I20260809 02:14:31.608371 9436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:50053
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:31.627763 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41469
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:31.737632 9797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:31.737886 9797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:31.737933 9797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:31.739218 9797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:31.739290 9797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:31.739336 9797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:31.741061 9797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41469
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38447
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:31.741354 9797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:31.741597 9797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:31.747285 9803 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.747308 9802 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:31.747352 9805 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:31.747538 9797 server_base.cc:1034] running on GCE node
I20260809 02:14:31.747758 9797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:31.748006 9797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:31.749166 9797 hybrid_clock.cc:648] HybridClock initialized: now 1786241671749156 us; error 34 us; skew 500 ppm
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:31.752388 9797 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:31.752956 9797 webserver.cc:460] Webserver started at http://127.9.40.67:45169/ using document root <none> and password file <none>
I20260809 02:14:31.753208 9797 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:31.753281 9797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:31.753432 9797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:31.754741 9797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "589e749b956d4f8aad4a23550020e702"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "6156119ca4c185db57e35844581342e7"
server_key_iv: "a937c5400c728f106f4d61f22e5b551f"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.755239 9797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "589e749b956d4f8aad4a23550020e702"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "6156119ca4c185db57e35844581342e7"
server_key_iv: "a937c5400c728f106f4d61f22e5b551f"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.756878 9797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 02:14:31.757670 9812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.757879 9797 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:31.757995 9797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "589e749b956d4f8aad4a23550020e702"
format_stamp: "Formatted at 2026-08-09 02:14:31 on dist-test-slave-qg60"
server_key: "6156119ca4c185db57e35844581342e7"
server_key_iv: "a937c5400c728f106f4d61f22e5b551f"
server_key_version: "encryptionkey@0"
I20260809 02:14:31.758082 9797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:31.789855 9797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:31.792268 9797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:31.792513 9797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:31.792990 9797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:31.793388 9797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:31.793452 9797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.793501 9797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:31.793529 9797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:31.806389 9797 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:45721
I20260809 02:14:31.806434 9937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:45721 every 8 connection(s)
I20260809 02:14:31.806825 9797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:31.814409 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 9797
I20260809 02:14:31.814499 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:31.814683 9377 external_mini_cluster.cc:1428] Setting key 4b7c3bb68eebaff17dc9726e723968cd
I20260809 02:14:31.817703 9500 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.807697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:38915 (local address 127.9.40.126:38447)
0809 02:14:31.807772 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.807774 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.808133 (+ 359us) server_negotiation.cc:407] Connection header received
0809 02:14:31.809038 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.809040 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.809054 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.809105 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.810264 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.810770 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.811959 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.812176 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.812254 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.814801 (+ 2547us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.814803 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.814803 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.814812 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.816263 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.816684 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.816685 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.816686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.816787 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.817044 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.817045 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.817045 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.817168 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.817254 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.817520 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.817600 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:14:31.817900 9941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.807666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38447 (local address 127.9.40.67:38915)
0809 02:14:31.807937 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:31.807977 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:31.808898 (+ 921us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:31.809183 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:31.809187 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:31.809834 (+ 647us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:31.810148 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.810153 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.810870 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.810871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.811862 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:31.811865 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.812274 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.812275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:31.812456 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.813068 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:31.813076 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:31.814700 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:31.816347 (+ 1647us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.816354 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.816356 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.816582 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.816860 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:31.816861 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:31.816861 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:31.816975 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:31.817235 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:31.817242 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:31.817389 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:14:31.817497 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:14:31.817662 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":153,"threads_started":1}
I20260809 02:14:31.818341 9938 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38447
I20260809 02:14:31.818428 9938 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:31.818570 9938 heartbeater.cc:511] Master 127.9.40.126:38447 requested a full tablet report, sending...
I20260809 02:14:31.818908 9436 ts_manager.cc:194] Registered new tserver with Master: 589e749b956d4f8aad4a23550020e702 (127.9.40.67:45721)
I20260809 02:14:31.819298 9436 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:38915
I20260809 02:14:31.826964 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:31.855796 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.845034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47248 (local address 127.9.40.65:45269)
0809 02:14:31.845426 (+ 392us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.845432 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.845447 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:14:31.845483 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.845485 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.845503 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.845658 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.846651 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.848179 (+ 1528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.849152 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.849553 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.849621 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.852215 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.852226 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.852226 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.852234 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.854401 (+ 2167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.854798 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.854802 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.854802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.854867 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.855133 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.855134 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.855134 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.855286 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.855445 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.855525 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.855616 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":115,"threads_started":1}
W20260809 02:14:31.856259 9552 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:47248
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9387](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241671, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:31.882845 9946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:31.874755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47264 (local address 127.9.40.65:45269)
0809 02:14:31.874823 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:31.874824 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:31.874854 (+ 30us) server_negotiation.cc:407] Connection header received
0809 02:14:31.874933 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:31.874934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:31.874946 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:31.874998 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:31.875679 (+ 681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.876257 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.877248 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:31.877455 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:31.877526 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:31.880217 (+ 2691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:31.880219 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:31.880219 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:31.880226 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:31.881763 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.882094 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.882095 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.882095 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.882152 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:31.882325 (+ 173us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:31.882326 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:31.882326 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:31.882420 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:31.882548 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:31.882669 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 02:14:31.882767 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:14:31.883538 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 9505
I20260809 02:14:31.890471 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 9651
I20260809 02:14:31.897622 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 9797
I20260809 02:14:31.903112 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 9402
2026-08-09T02:14:31Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2178 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9953](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9953](info): set up 2 sockets
Aug 09 02:14:31 dist-test-slave-qg60 krb5kdc[9953](info): commencing operation
krb5kdc: starting...
W20260809 02:14:33.013321 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:33Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:33Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:33.146293 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45801
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45801
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:33.231264 9968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:33.231537 9968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:33.231576 9968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:33.232832 9968 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:33.232916 9968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:33.232944 9968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:33.232966 9968 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:33.232996 9968 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:33.234588 9968 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45801
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45801
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:33.234891 9968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:33.235144 9968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:33.240460 9976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.240499 9974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.240466 9973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:33.240680 9968 server_base.cc:1034] running on GCE node
I20260809 02:14:33.240954 9968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:33.241279 9968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:33.242451 9968 hybrid_clock.cc:648] HybridClock initialized: now 1786241673242444 us; error 54 us; skew 500 ppm
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:33.245184 9968 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:33.245720 9968 webserver.cc:460] Webserver started at http://127.9.40.126:44351/ using document root <none> and password file <none>
I20260809 02:14:33.245923 9968 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:33.245965 9968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:33.246053 9968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:33.247113 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "4cdfef83cb9a4fc992478b82069410f8"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "4aa45796a6ba0bb442c2829bdb6bd9e6"
server_key_iv: "4932a9366d18829497328810138d18a9"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.247431 9968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "4cdfef83cb9a4fc992478b82069410f8"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "4aa45796a6ba0bb442c2829bdb6bd9e6"
server_key_iv: "4932a9366d18829497328810138d18a9"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.248867 9968 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:33.249785 9983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.250028 9968 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:33.250154 9968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "4cdfef83cb9a4fc992478b82069410f8"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "4aa45796a6ba0bb442c2829bdb6bd9e6"
server_key_iv: "4932a9366d18829497328810138d18a9"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.250221 9968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:33.266255 9968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:33.268630 9968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:33.268838 9968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:33.272987 9968 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45801
I20260809 02:14:33.273062 10039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45801 every 8 connection(s)
I20260809 02:14:33.273429 9968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:33.274361 10040 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:33.276821 10040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: Bootstrap starting.
I20260809 02:14:33.277426 10040 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:33.277742 10040 log.cc:824] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:33.278604 10040 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: No bootstrap required, opened a new log
I20260809 02:14:33.280236 10040 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } }
I20260809 02:14:33.280402 10040 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:33.280450 10040 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4cdfef83cb9a4fc992478b82069410f8, State: Initialized, Role: FOLLOWER
I20260809 02:14:33.280647 10040 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } }
I20260809 02:14:33.280779 10040 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:33.280823 10040 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:33.280876 10040 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:33.281435 10040 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } }
I20260809 02:14:33.281558 10040 leader_election.cc:302] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4cdfef83cb9a4fc992478b82069410f8; no voters:
I20260809 02:14:33.281754 10040 leader_election.cc:288] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:33.281801 10043 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:33.281996 10043 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [term 1 LEADER]: Becoming Leader. State: Replica: 4cdfef83cb9a4fc992478b82069410f8, State: Running, Role: LEADER
I20260809 02:14:33.282037 10040 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:33.282276 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 9968
I20260809 02:14:33.282377 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:33.282547 9377 external_mini_cluster.cc:1428] Setting key 608e7dbc8c90219e68e8a8b1f141f3cc
I20260809 02:14:33.282649 10043 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } }
I20260809 02:14:33.283399 10045 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4cdfef83cb9a4fc992478b82069410f8. Latest consensus state: current_term: 1 leader_uuid: "4cdfef83cb9a4fc992478b82069410f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } } }
I20260809 02:14:33.283504 10045 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:33.283742 10044 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4cdfef83cb9a4fc992478b82069410f8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4cdfef83cb9a4fc992478b82069410f8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45801 } } }
I20260809 02:14:33.283998 10044 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8 [sys.catalog]: This master's current role is: LEADER
W20260809 02:14:33.284085 10061 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:33.284147 10061 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:33.284207 10062 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:33.284363 10062 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:33.286063 10062 catalog_manager.cc:1347] Generated new cluster ID: 241349b55c434b0ea5e89d6db0cd30ac
I20260809 02:14:33.286114 10062 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:33.294943 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.283047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50240 (local address 127.9.40.126:45801)
0809 02:14:33.283928 (+ 881us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.283938 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.283975 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:14:33.284100 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.284107 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.284164 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.284308 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.285128 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.286406 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.287445 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.288024 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.288407 (+ 383us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.290891 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.290895 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.290895 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.290907 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.292992 (+ 2085us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.293381 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.293384 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.293384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.293509 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.293691 (+ 182us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.293692 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.293692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.293983 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.294502 (+ 519us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.294568 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.294711 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":833,"thread_start_us":102,"threads_started":1}
I20260809 02:14:33.308660 10062 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:33.309280 10062 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:33.312393 10062 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4cdfef83cb9a4fc992478b82069410f8: Generated new TSK 0
I20260809 02:14:33.312589 10062 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:33.344151 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:33.427747 10068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:33.427963 10068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:33.427989 10068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:33.429291 10068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:33.429337 10068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:33.429364 10068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:33.430892 10068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:33.431149 10068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:33.431406 10068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:33.436784 10073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.436770 10074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.436797 10076 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:33.436937 10068 server_base.cc:1034] running on GCE node
I20260809 02:14:33.437274 10068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:33.437534 10068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:33.438730 10068 hybrid_clock.cc:648] HybridClock initialized: now 1786241673438709 us; error 38 us; skew 500 ppm
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:33.441545 10068 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:33.442003 10068 webserver.cc:460] Webserver started at http://127.9.40.65:37865/ using document root <none> and password file <none>
I20260809 02:14:33.442301 10068 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:33.442373 10068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:33.442560 10068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:33.443643 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "1633be9deaae4321826d514baf2e0bd1"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "1a592773df4fdee0d4e515ba48e94c6b"
server_key_iv: "db553e3053664f9d1154e2cd83c9e992"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.444021 10068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "1633be9deaae4321826d514baf2e0bd1"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "1a592773df4fdee0d4e515ba48e94c6b"
server_key_iv: "db553e3053664f9d1154e2cd83c9e992"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.445474 10068 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:33.446235 10083 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.446412 10068 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:33.446524 10068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "1633be9deaae4321826d514baf2e0bd1"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "1a592773df4fdee0d4e515ba48e94c6b"
server_key_iv: "db553e3053664f9d1154e2cd83c9e992"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.446602 10068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:33.465183 10068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:33.467588 10068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:33.467757 10068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:33.468011 10068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:33.468343 10068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:33.468398 10068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.468442 10068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:33.468468 10068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.474479 10068 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:35633
I20260809 02:14:33.474584 10208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:35633 every 8 connection(s)
I20260809 02:14:33.474895 10068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:33.479959 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10068
I20260809 02:14:33.480067 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:33.480221 9377 external_mini_cluster.cc:1428] Setting key 30730d59f565f4cafecf3f9062c36641
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:33.486613 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.475717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53203 (local address 127.9.40.126:45801)
0809 02:14:33.475771 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.475773 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.476022 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:14:33.477171 (+ 1149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.477172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.477188 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.477245 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.477893 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.478412 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.480006 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.480364 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.480448 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.483235 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.483237 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.483237 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.483246 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.484916 (+ 1670us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.485472 (+ 556us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.485473 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.485473 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.485538 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.485872 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.485873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.485874 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.486009 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.486078 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.486445 (+ 367us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.486539 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:33.486927 10212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.475711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45801 (local address 127.9.40.65:53203)
0809 02:14:33.475892 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.475929 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.477068 (+ 1139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.477325 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.477328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.477617 (+ 289us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:33.477814 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.477818 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.478498 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.478498 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.479893 (+ 1395us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.479896 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.480495 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.480496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.480654 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.481231 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:33.481242 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:33.483133 (+ 1891us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:33.485000 (+ 1867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.485008 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.485011 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.485363 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.485623 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.485625 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.485625 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.485792 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.486089 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:33.486096 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:33.486279 (+ 183us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.486406 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.486601 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":94,"threads_started":1}
I20260809 02:14:33.487550 10210 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45801
I20260809 02:14:33.487676 10210 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:33.487883 10210 heartbeater.cc:511] Master 127.9.40.126:45801 requested a full tablet report, sending...
I20260809 02:14:33.488464 10000 ts_manager.cc:194] Registered new tserver with Master: 1633be9deaae4321826d514baf2e0bd1 (127.9.40.65:35633)
I20260809 02:14:33.489171 10000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:53203
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:33.511307 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:33.597064 10215 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:33.597283 10215 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:33.597311 10215 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:33.598435 10215 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:33.598472 10215 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:33.598500 10215 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:33.599931 10215 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10215
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:33.600124 10215 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:33.600303 10215 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:33.605423 10223 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.605427 10220 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.605495 10221 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:33.605654 10215 server_base.cc:1034] running on GCE node
I20260809 02:14:33.605918 10215 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:33.606220 10215 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:33.607386 10215 hybrid_clock.cc:648] HybridClock initialized: now 1786241673607369 us; error 42 us; skew 500 ppm
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:33.610232 10215 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:33.610661 10215 webserver.cc:460] Webserver started at http://127.9.40.66:39793/ using document root <none> and password file <none>
I20260809 02:14:33.610896 10215 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:33.610942 10215 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:33.611100 10215 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:33.612210 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "922c9588863f41aa88fcde361f3f0cd0"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "09c27740da243ccddd4b8bff48d879ca"
server_key_iv: "0fc63e89edc51ed3c4364e4d0fca43cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.612587 10215 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "922c9588863f41aa88fcde361f3f0cd0"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "09c27740da243ccddd4b8bff48d879ca"
server_key_iv: "0fc63e89edc51ed3c4364e4d0fca43cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.614003 10215 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:33.614763 10232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.614949 10215 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:14:33.615042 10215 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "922c9588863f41aa88fcde361f3f0cd0"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "09c27740da243ccddd4b8bff48d879ca"
server_key_iv: "0fc63e89edc51ed3c4364e4d0fca43cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.615123 10215 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:33.627398 10215 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:33.629868 10215 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:33.630030 10215 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:33.630293 10215 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:33.630621 10215 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:33.630684 10215 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.630728 10215 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:33.630756 10215 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.636998 10215 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:37317
I20260809 02:14:33.637073 10360 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:37317 every 8 connection(s)
I20260809 02:14:33.637432 10215 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:33.647076 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10215
I20260809 02:14:33.647210 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:33.647404 9377 external_mini_cluster.cc:1428] Setting key 23e85d6af00e16e7f761a1d562f253e0
I20260809 02:14:33.647892 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.638270 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50061 (local address 127.9.40.126:45801)
0809 02:14:33.638369 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.638372 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.638572 (+ 200us) server_negotiation.cc:407] Connection header received
0809 02:14:33.639841 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.639842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.639859 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.639958 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.640810 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.641365 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.642383 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.642586 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.642645 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.644895 (+ 2250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.644896 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.644897 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.644904 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.646394 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.646810 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.646811 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.646811 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.646887 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.647128 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.647129 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.647130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.647294 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.647383 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.647690 (+ 307us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.647808 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:14:33.648113 10364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.638190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45801 (local address 127.9.40.66:50061)
0809 02:14:33.638430 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.638479 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.639691 (+ 1212us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.640124 (+ 433us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.640129 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.640463 (+ 334us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:33.640666 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.640671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.641452 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.641453 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.642301 (+ 848us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.642303 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.642671 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.642672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.642772 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.643283 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:33.643291 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:33.644783 (+ 1492us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:33.646478 (+ 1695us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.646486 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.646487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.646740 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.646956 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.646957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.646957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.647057 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.647372 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:33.647378 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:33.647550 (+ 172us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.647654 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.647853 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":101,"threads_started":1}
I20260809 02:14:33.648589 10361 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45801
I20260809 02:14:33.648703 10361 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:33.648954 10361 heartbeater.cc:511] Master 127.9.40.126:45801 requested a full tablet report, sending...
I20260809 02:14:33.649405 10000 ts_manager.cc:194] Registered new tserver with Master: 922c9588863f41aa88fcde361f3f0cd0 (127.9.40.66:37317)
I20260809 02:14:33.649914 10000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:50061
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:33.680334 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:34551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:33.765983 10367 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:33.766181 10367 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:33.766206 10367 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:33.767294 10367 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:33.767330 10367 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:33.767356 10367 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:33.768836 10367 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45801
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10367
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:33.769034 10367 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:33.769208 10367 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:33.774260 10377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.774231 10374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:33.774236 10372 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:33.774458 10367 server_base.cc:1034] running on GCE node
I20260809 02:14:33.774771 10367 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:33.774981 10367 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:33.776206 10367 hybrid_clock.cc:648] HybridClock initialized: now 1786241673776192 us; error 33 us; skew 500 ppm
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:33.778975 10367 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:33.779462 10367 webserver.cc:460] Webserver started at http://127.9.40.67:40007/ using document root <none> and password file <none>
I20260809 02:14:33.779697 10367 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:33.779765 10367 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:33.779919 10367 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:33.781070 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "3e636d7eddb44a76b2ded2fbb59cc02e"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "485d1d9098948940c5b1f1dda6b04687"
server_key_iv: "fbefadeb22730d38a37e06a213217880"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.781441 10367 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "3e636d7eddb44a76b2ded2fbb59cc02e"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "485d1d9098948940c5b1f1dda6b04687"
server_key_iv: "fbefadeb22730d38a37e06a213217880"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.783066 10367 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:33.783983 10385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.784237 10367 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:33.784332 10367 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "3e636d7eddb44a76b2ded2fbb59cc02e"
format_stamp: "Formatted at 2026-08-09 02:14:33 on dist-test-slave-qg60"
server_key: "485d1d9098948940c5b1f1dda6b04687"
server_key_iv: "fbefadeb22730d38a37e06a213217880"
server_key_version: "encryptionkey@0"
I20260809 02:14:33.784421 10367 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:33.801723 10367 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:33.804163 10367 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:33.804338 10367 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:33.804585 10367 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:33.804981 10367 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:33.805037 10367 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.805089 10367 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:33.805115 10367 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:33.811209 10367 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44985
I20260809 02:14:33.811277 10518 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44985 every 8 connection(s)
I20260809 02:14:33.811622 10367 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:33.816673 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10367
I20260809 02:14:33.816803 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:33.817072 9377 external_mini_cluster.cc:1428] Setting key 627737bab2bea36aef9bdbf78c9a6cad
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:33.821748 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.812597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60109 (local address 127.9.40.126:45801)
0809 02:14:33.812657 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.812659 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.812914 (+ 255us) server_negotiation.cc:407] Connection header received
0809 02:14:33.813873 (+ 959us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.813874 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.813886 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.813937 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.814550 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.815108 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.816121 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.816297 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.816350 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.818951 (+ 2601us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.818952 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.818952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.818959 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.820461 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.820936 (+ 475us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.820937 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.820937 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.820985 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.821215 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.821216 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.821216 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.821310 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.821355 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.821624 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.821684 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:14:33.822016 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.812570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45801 (local address 127.9.40.67:60109)
0809 02:14:33.812779 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.812828 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.813775 (+ 947us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.814001 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.814004 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.814276 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:33.814475 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.814479 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.815193 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.815193 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.816038 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.816040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.816373 (+ 333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.816374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.816524 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.817095 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:33.817105 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:33.818843 (+ 1738us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:33.820549 (+ 1706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.820556 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.820557 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.820871 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.821052 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:33.821053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:33.821053 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:33.821168 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:33.821363 (+ 195us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:33.821367 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:33.821487 (+ 120us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.821590 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.821722 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260809 02:14:33.822589 10519 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45801
I20260809 02:14:33.822678 10519 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:33.822887 10519 heartbeater.cc:511] Master 127.9.40.126:45801 requested a full tablet report, sending...
I20260809 02:14:33.823333 10000 ts_manager.cc:194] Registered new tserver with Master: 3e636d7eddb44a76b2ded2fbb59cc02e (127.9.40.67:44985)
I20260809 02:14:33.823760 10000 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60109
I20260809 02:14:33.829131 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:14:33.837158 10053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.830900 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47894 (local address 127.9.40.126:45801)
0809 02:14:33.830954 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.830956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.831050 (+ 94us) server_negotiation.cc:407] Connection header received
0809 02:14:33.831108 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.831108 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.831117 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.831199 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.831743 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.832315 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.833191 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.833417 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.833482 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.834714 (+ 1232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.834715 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.834715 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.834721 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.835989 (+ 1268us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.836337 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.836338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.836338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.836393 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.836576 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.836576 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.836577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.836800 (+ 223us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.836881 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.837005 (+ 124us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.837088 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:33.839601 10000 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47894:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:33.841481 10000 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:33.848881 10532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44985 (local address 127.0.0.1:33444)
0809 02:14:33.844348 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.844412 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.844487 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.844981 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.844984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.844986 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:33.845074 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.845077 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846395 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.846396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.847501 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.847504 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.848446 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.848447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.848554 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.848572 (+ 18us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.848691 (+ 119us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.848765 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":88,"thread_start_us":61,"threads_started":1}
I20260809 02:14:33.848953 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844296 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42902 (local address 127.9.40.66:37317)
0809 02:14:33.844584 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.844590 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.844849 (+ 259us) server_negotiation.cc:407] Connection header received
0809 02:14:33.844918 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.844920 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.844936 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.845007 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:33.845443 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846137 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.847751 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.848455 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.848523 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.848584 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.848703 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.848827 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":248,"thread_start_us":84,"threads_started":1}
I20260809 02:14:33.848925 10534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844158 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:37317 (local address 127.0.0.1:42902)
0809 02:14:33.844760 (+ 602us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.844772 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.844841 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.845338 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.845339 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.845341 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:33.845383 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.845384 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846257 (+ 873us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.846259 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.847656 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.847660 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.848536 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.848536 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.848587 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.848590 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.848667 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.848767 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":551,"mutex_wait_us":32,"thread_start_us":116,"threads_started":1}
I20260809 02:14:33.849426 10536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33444 (local address 127.9.40.67:44985)
0809 02:14:33.844780 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.844784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.844794 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:33.844855 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.844857 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.844876 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.844965 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:33.845141 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846288 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.847595 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.848352 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.848440 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.848514 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.849108 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.849158 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":112,"threads_started":1}
I20260809 02:14:33.849952 10531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:35633 (local address 127.0.0.1:59054)
0809 02:14:33.844770 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.844780 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.844841 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.845253 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.845254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.845256 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:33.845306 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.845308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846562 (+ 1254us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.846563 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.847659 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.847660 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.849709 (+ 2049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.849710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.849775 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.849780 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.849840 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.849879 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":359,"thread_start_us":75,"threads_started":1}
I20260809 02:14:33.850065 10285 tablet_service.cc:1467] Processing CreateTablet for tablet 4360c2605a9f482e9af4b3d1f7d4ddee (DEFAULT_TABLE table=test-table [id=81f620119b414b57a0e512ea18e19fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:33.850229 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.844543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59054 (local address 127.9.40.65:35633)
0809 02:14:33.844766 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.844771 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.845095 (+ 324us) server_negotiation.cc:407] Connection header received
0809 02:14:33.845153 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.845155 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.845171 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.845245 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:33.845370 (+ 125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.846429 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.848658 (+ 2229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.849616 (+ 958us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.849700 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.849763 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.850106 (+ 343us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.850154 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":86,"threads_started":1}
I20260809 02:14:33.850203 10440 tablet_service.cc:1467] Processing CreateTablet for tablet 4360c2605a9f482e9af4b3d1f7d4ddee (DEFAULT_TABLE table=test-table [id=81f620119b414b57a0e512ea18e19fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:33.850479 10285 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4360c2605a9f482e9af4b3d1f7d4ddee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:33.850670 10440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4360c2605a9f482e9af4b3d1f7d4ddee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:33.851154 10132 tablet_service.cc:1467] Processing CreateTablet for tablet 4360c2605a9f482e9af4b3d1f7d4ddee (DEFAULT_TABLE table=test-table [id=81f620119b414b57a0e512ea18e19fa5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:33.851485 10132 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4360c2605a9f482e9af4b3d1f7d4ddee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:33.856019 10537 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: Bootstrap starting.
I20260809 02:14:33.856812 10539 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Bootstrap starting.
I20260809 02:14:33.858440 10537 tablet_bootstrap.cc:654] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:33.858434 10539 tablet_bootstrap.cc:654] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:33.859998 10539 log.cc:824] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:33.860039 10537 log.cc:824] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:33.860514 10538 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Bootstrap starting.
I20260809 02:14:33.861184 10537 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: No bootstrap required, opened a new log
I20260809 02:14:33.861299 10537 ts_tablet_manager.cc:1397] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:14:33.861747 10539 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: No bootstrap required, opened a new log
I20260809 02:14:33.861825 10539 ts_tablet_manager.cc:1397] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:14:33.861950 10538 tablet_bootstrap.cc:654] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:33.862828 10538 log.cc:824] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:33.863760 10538 tablet_bootstrap.cc:492] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: No bootstrap required, opened a new log
I20260809 02:14:33.863849 10538 ts_tablet_manager.cc:1397] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Time spent bootstrapping tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:33.866199 10537 raft_consensus.cc:348] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.866531 10537 raft_consensus.cc:374] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:33.866609 10537 raft_consensus.cc:729] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e636d7eddb44a76b2ded2fbb59cc02e, State: Initialized, Role: FOLLOWER
I20260809 02:14:33.866750 10537 consensus_queue.cc:260] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.866734 10539 raft_consensus.cc:348] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.867079 10539 raft_consensus.cc:374] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:33.867132 10539 raft_consensus.cc:729] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1633be9deaae4321826d514baf2e0bd1, State: Initialized, Role: FOLLOWER
I20260809 02:14:33.867267 10537 ts_tablet_manager.cc:1428] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e: Time spent starting tablet: real 0.006s user 0.000s sys 0.006s
I20260809 02:14:33.867313 10539 consensus_queue.cc:260] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.867149 10538 raft_consensus.cc:348] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.867441 10519 heartbeater.cc:503] Master 127.9.40.126:45801 was elected leader, sending a full tablet report...
I20260809 02:14:33.867486 10538 raft_consensus.cc:374] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:33.867524 10538 raft_consensus.cc:729] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 922c9588863f41aa88fcde361f3f0cd0, State: Initialized, Role: FOLLOWER
I20260809 02:14:33.867682 10538 consensus_queue.cc:260] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.867790 10539 ts_tablet_manager.cc:1428] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Time spent starting tablet: real 0.006s user 0.001s sys 0.005s
I20260809 02:14:33.867919 10210 heartbeater.cc:503] Master 127.9.40.126:45801 was elected leader, sending a full tablet report...
I20260809 02:14:33.868211 10538 ts_tablet_manager.cc:1428] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260809 02:14:33.868242 10361 heartbeater.cc:503] Master 127.9.40.126:45801 was elected leader, sending a full tablet report...
W20260809 02:14:33.888167 10363 tablet.cc:2369] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:33.913507 10543 raft_consensus.cc:482] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:33.913702 10543 raft_consensus.cc:504] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.914103 10543 leader_election.cc:288] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 922c9588863f41aa88fcde361f3f0cd0 (127.9.40.66:37317), 1633be9deaae4321826d514baf2e0bd1 (127.9.40.65:35633)
I20260809 02:14:33.918342 10521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.914434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:37317 (local address 127.9.40.67:53071)
0809 02:14:33.914538 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.914549 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.914607 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.914928 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.914929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.914931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:33.915053 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.915055 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.916013 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.916014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.917099 (+ 1085us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.917102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.917941 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.917942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.918102 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.918107 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.918175 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.918215 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81}
I20260809 02:14:33.918345 10533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.914637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53071 (local address 127.9.40.66:37317)
0809 02:14:33.914732 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.914735 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.914745 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:33.914774 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.914791 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.914805 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.914869 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:33.915176 (+ 307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.915922 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.917225 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.917849 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.917927 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.917938 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.918192 (+ 254us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.918265 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:14:33.918833 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4360c2605a9f482e9af4b3d1f7d4ddee" candidate_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c9588863f41aa88fcde361f3f0cd0" is_pre_election: true
I20260809 02:14:33.919092 10308 raft_consensus.cc:2454] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e636d7eddb44a76b2ded2fbb59cc02e in term 0.
I20260809 02:14:33.919359 10387 leader_election.cc:302] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e636d7eddb44a76b2ded2fbb59cc02e, 922c9588863f41aa88fcde361f3f0cd0; no voters:
I20260809 02:14:33.919490 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.914847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43649 (local address 127.9.40.65:35633)
0809 02:14:33.914901 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.914903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.915123 (+ 220us) server_negotiation.cc:407] Connection header received
0809 02:14:33.915201 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.915202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.915212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.915252 (+ 40us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:33.915623 (+ 371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.916256 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.918139 (+ 1883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.918887 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.918948 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.918956 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.919366 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.919414 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:14:33.919533 10543 raft_consensus.cc:2781] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:33.919606 10543 raft_consensus.cc:482] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:33.919518 10547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.914635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:35633 (local address 127.9.40.67:43649)
0809 02:14:33.915027 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:33.915038 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:33.915103 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:33.915283 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:33.915284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:33.915286 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:33.915526 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.915528 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.916362 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.916363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.918045 (+ 1682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:33.918048 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.919142 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.919143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:33.919277 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.919281 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:33.919340 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:14:33.919369 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":215,"threads_started":1}
I20260809 02:14:33.919677 10543 raft_consensus.cc:3037] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:33.920079 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4360c2605a9f482e9af4b3d1f7d4ddee" candidate_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1633be9deaae4321826d514baf2e0bd1" is_pre_election: true
I20260809 02:14:33.920271 10156 raft_consensus.cc:2454] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3e636d7eddb44a76b2ded2fbb59cc02e in term 0.
I20260809 02:14:33.921073 10543 raft_consensus.cc:504] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.921190 10543 leader_election.cc:288] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [CANDIDATE]: Term 1 election: Requested vote from peers 922c9588863f41aa88fcde361f3f0cd0 (127.9.40.66:37317), 1633be9deaae4321826d514baf2e0bd1 (127.9.40.65:35633)
I20260809 02:14:33.921475 10156 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4360c2605a9f482e9af4b3d1f7d4ddee" candidate_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1633be9deaae4321826d514baf2e0bd1"
I20260809 02:14:33.921582 10156 raft_consensus.cc:3037] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:33.921589 10308 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4360c2605a9f482e9af4b3d1f7d4ddee" candidate_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "922c9588863f41aa88fcde361f3f0cd0"
I20260809 02:14:33.921696 10308 raft_consensus.cc:3037] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:33.922870 10156 raft_consensus.cc:2454] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e636d7eddb44a76b2ded2fbb59cc02e in term 1.
I20260809 02:14:33.923046 10387 leader_election.cc:302] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1633be9deaae4321826d514baf2e0bd1, 3e636d7eddb44a76b2ded2fbb59cc02e; no voters:
I20260809 02:14:33.923139 10543 raft_consensus.cc:2781] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:33.923241 10308 raft_consensus.cc:2454] T 4360c2605a9f482e9af4b3d1f7d4ddee P 922c9588863f41aa88fcde361f3f0cd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3e636d7eddb44a76b2ded2fbb59cc02e in term 1.
I20260809 02:14:33.923408 10543 raft_consensus.cc:686] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [term 1 LEADER]: Becoming Leader. State: Replica: 3e636d7eddb44a76b2ded2fbb59cc02e, State: Running, Role: LEADER
I20260809 02:14:33.923520 10543 consensus_queue.cc:237] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } }
I20260809 02:14:33.924547 10000 catalog_manager.cc:5495] T 4360c2605a9f482e9af4b3d1f7d4ddee P 3e636d7eddb44a76b2ded2fbb59cc02e reported cstate change: term changed from 0 to 1, leader changed from <none> to 3e636d7eddb44a76b2ded2fbb59cc02e (127.9.40.67). New cstate: current_term: 1 leader_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e636d7eddb44a76b2ded2fbb59cc02e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44985 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "922c9588863f41aa88fcde361f3f0cd0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 37317 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1633be9deaae4321826d514baf2e0bd1" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35633 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:33.950179 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.942060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59066 (local address 127.9.40.65:35633)
0809 02:14:33.942132 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.942134 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.942293 (+ 159us) server_negotiation.cc:407] Connection header received
0809 02:14:33.942378 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.942380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.942392 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.942466 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.943217 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.943718 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.944519 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.944777 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.944847 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.947362 (+ 2515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.947377 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.947377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.947394 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.948903 (+ 1509us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.949341 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.949345 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.949346 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.949408 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.949613 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.949615 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.949615 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.949781 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.949951 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.949989 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.950072 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:33.974504 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.967249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59074 (local address 127.9.40.65:35633)
0809 02:14:33.967322 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.967325 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.967418 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:14:33.967519 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.967520 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.967529 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.967584 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.968192 (+ 608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.968676 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.969465 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.969655 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.969711 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.972016 (+ 2305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.972018 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.972018 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.972026 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.973494 (+ 1468us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.973817 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.973817 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.973818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.973867 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.974030 (+ 163us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.974031 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.974031 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.974146 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.974216 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.974371 (+ 155us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.974437 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
W20260809 02:14:33.975662 10211 tablet.cc:2369] T 4360c2605a9f482e9af4b3d1f7d4ddee P 1633be9deaae4321826d514baf2e0bd1: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:14:33 dist-test-slave-qg60 krb5kdc[9953](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241673, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:33.998937 10535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:33.991516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59090 (local address 127.9.40.65:35633)
0809 02:14:33.991608 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:33.991611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:33.991718 (+ 107us) server_negotiation.cc:407] Connection header received
0809 02:14:33.991808 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:33.991810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:33.991828 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:33.991892 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:33.992456 (+ 564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.993004 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.993784 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:33.993973 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:33.994050 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:33.996333 (+ 2283us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:33.996335 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:33.996335 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:33.996350 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:33.997661 (+ 1311us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.998078 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.998079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.998080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.998148 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:33.998365 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:33.998366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:33.998366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:33.998485 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:33.998595 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:33.998744 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 02:14:33.998828 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 02:14:33.999433 10112 tablet_service.cc:2908] Rejecting scan request for tablet 4360c2605a9f482e9af4b3d1f7d4ddee: Uninitialized: clean time has not yet been initialized
I20260809 02:14:34.000357 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10068
I20260809 02:14:34.010871 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10215
I20260809 02:14:34.017379 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10367
I20260809 02:14:34.025641 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 9968
2026-08-09T02:14:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2120 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:34 dist-test-slave-qg60 krb5kdc[10565](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:34 dist-test-slave-qg60 krb5kdc[10565](info): set up 2 sockets
Aug 09 02:14:34 dist-test-slave-qg60 krb5kdc[10565](info): commencing operation
krb5kdc: starting...
W20260809 02:14:35.201892 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.153s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:35.335959 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:32899
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:32899
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:35.420228 10580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:35.420478 10580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:35.420516 10580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:35.421784 10580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:35.421857 10580 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:14:35.421880 10580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:35.421902 10580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:35.421943 10580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:35.421972 10580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:35.423488 10580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:32899
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:32899
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:35.423770 10580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:35.424026 10580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:35.429044 10585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.429064 10588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.429052 10586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.429111 10580 server_base.cc:1034] running on GCE node
I20260809 02:14:35.429697 10580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:35.430030 10580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:35.431214 10580 hybrid_clock.cc:648] HybridClock initialized: now 1786241675431215 us; error 45 us; skew 500 ppm
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:35.439020 10580 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:35.439528 10580 webserver.cc:460] Webserver started at http://127.9.40.126:37013/ using document root <none> and password file <none>
I20260809 02:14:35.439745 10580 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:35.439787 10580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:35.439880 10580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:35.441002 10580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "bf35738cafa44aae98853a690b765b74"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "028617e69f67b950721fa299ed2efe6e"
server_key_iv: "56272f4c0735843039cc01d415fe0597"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.441306 10580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bf35738cafa44aae98853a690b765b74"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "028617e69f67b950721fa299ed2efe6e"
server_key_iv: "56272f4c0735843039cc01d415fe0597"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.442646 10580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:35.443528 10596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.443766 10580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:35.443861 10580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "bf35738cafa44aae98853a690b765b74"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "028617e69f67b950721fa299ed2efe6e"
server_key_iv: "56272f4c0735843039cc01d415fe0597"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.443928 10580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:35.464102 10580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:35.466590 10580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:35.466709 10580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:35.471540 10580 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:32899
I20260809 02:14:35.471666 10666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:32899 every 8 connection(s)
I20260809 02:14:35.471997 10580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:35.472838 10667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:35.475111 10667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Bootstrap starting.
I20260809 02:14:35.475723 10667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:35.476018 10667 log.cc:824] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:35.476775 10667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: No bootstrap required, opened a new log
I20260809 02:14:35.478945 10667 raft_consensus.cc:348] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } }
I20260809 02:14:35.479112 10667 raft_consensus.cc:374] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:35.479152 10667 raft_consensus.cc:729] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bf35738cafa44aae98853a690b765b74, State: Initialized, Role: FOLLOWER
I20260809 02:14:35.479346 10667 consensus_queue.cc:260] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } }
I20260809 02:14:35.479451 10667 raft_consensus.cc:388] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:35.479492 10667 raft_consensus.cc:482] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:35.479545 10667 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:35.480108 10667 raft_consensus.cc:504] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } }
I20260809 02:14:35.480240 10667 leader_election.cc:302] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bf35738cafa44aae98853a690b765b74; no voters:
I20260809 02:14:35.480432 10667 leader_election.cc:288] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:35.480484 10670 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:35.480685 10670 raft_consensus.cc:686] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [term 1 LEADER]: Becoming Leader. State: Replica: bf35738cafa44aae98853a690b765b74, State: Running, Role: LEADER
I20260809 02:14:35.480746 10667 sys_catalog.cc:564] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:35.480810 10670 consensus_queue.cc:237] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } }
I20260809 02:14:35.481236 10671 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bf35738cafa44aae98853a690b765b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } } }
I20260809 02:14:35.481267 10672 sys_catalog.cc:455] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bf35738cafa44aae98853a690b765b74. Latest consensus state: current_term: 1 leader_uuid: "bf35738cafa44aae98853a690b765b74" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bf35738cafa44aae98853a690b765b74" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 32899 } } }
I20260809 02:14:35.481433 10671 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:35.481446 10672 sys_catalog.cc:458] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:35.481755 10680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:35.482169 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10580
I20260809 02:14:35.482272 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:35.482357 10680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:35.482436 9377 external_mini_cluster.cc:1428] Setting key 28ac3dccb54d937a583588b3c704d444
I20260809 02:14:35.483978 10680 catalog_manager.cc:1347] Generated new cluster ID: 759e9afdac53436c94779bc7c07db18a
I20260809 02:14:35.484032 10680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:35.494475 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.482929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56110 (local address 127.9.40.126:32899)
0809 02:14:35.483410 (+ 481us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:35.483418 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:35.483468 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:14:35.483605 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:35.483612 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:35.483672 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:35.483809 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:35.484507 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.485883 (+ 1376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.486923 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.487559 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.487688 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.490647 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:35.490651 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:35.490652 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:35.490664 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:35.492689 (+ 2025us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.493166 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.493169 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.493170 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.493291 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.493517 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.493518 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.493518 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.493833 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:35.493970 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:35.494108 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:14:35.494250 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":425,"thread_start_us":83,"threads_started":1}
I20260809 02:14:35.505676 10680 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:35.506232 10680 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:35.514357 10680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Generated new TSK 0
I20260809 02:14:35.514564 10680 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:35.552098 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:32899
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:35.662570 10698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:35.662833 10698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:35.662878 10698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:35.664482 10698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:35.664556 10698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:35.664593 10698 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:35.664634 10698 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:35.666265 10698 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:32899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:35.666559 10698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:35.666810 10698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:35.672132 10707 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.672199 10698 server_base.cc:1034] running on GCE node
W20260809 02:14:35.672147 10703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.672202 10704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.672588 10698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:35.672875 10698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:35.674028 10698 hybrid_clock.cc:648] HybridClock initialized: now 1786241675673994 us; error 49 us; skew 500 ppm
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:35.676667 10698 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:35.677208 10698 webserver.cc:460] Webserver started at http://127.9.40.65:37057/ using document root <none> and password file <none>
I20260809 02:14:35.677444 10698 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:35.677491 10698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:35.677670 10698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:35.678830 10698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "3de2893b0a25a1615fddcf4c59428a02"
server_key_iv: "791f62acf69a31fec4054121b78a5434"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.679212 10698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "3de2893b0a25a1615fddcf4c59428a02"
server_key_iv: "791f62acf69a31fec4054121b78a5434"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.680600 10698 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:35.681689 10714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.682010 10698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:35.682111 10698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "3de2893b0a25a1615fddcf4c59428a02"
server_key_iv: "791f62acf69a31fec4054121b78a5434"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.682224 10698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:35.695811 10698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:35.698385 10698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:35.698577 10698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:35.699122 10698 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:35.699159 10698 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.699205 10698 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:35.699235 10698 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.706507 10698 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45249
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:35.706589 10841 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45249 every 8 connection(s)
I20260809 02:14:35.706899 10698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:35.708650 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10698
I20260809 02:14:35.708798 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:35.709018 9377 external_mini_cluster.cc:1428] Setting key 17c8a311200f8b4b75f7e5667368a028
I20260809 02:14:35.710144 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.699802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51933 (local address 127.9.40.126:32899)
0809 02:14:35.699887 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:35.699890 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:35.700095 (+ 205us) server_negotiation.cc:407] Connection header received
0809 02:14:35.700661 (+ 566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:35.700663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:35.700681 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:35.700778 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:35.701738 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.702599 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.704251 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.704603 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.704693 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.707081 (+ 2388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:35.707083 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:35.707083 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:35.707094 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:35.708410 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.708940 (+ 530us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.708942 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.708943 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.709016 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.709377 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.709378 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.709379 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.709511 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:35.709586 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:35.709945 (+ 359us) server_negotiation.cc:299] Negotiation successful
0809 02:14:35.710043 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:14:35.710294 10725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.699748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.65:51933)
0809 02:14:35.699954 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:35.699996 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:35.700535 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:35.700872 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:35.700876 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:35.701304 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:35.701618 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.701624 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.702714 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.702715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.704128 (+ 1413us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.704131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.704703 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.704704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.704872 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.705452 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:35.705462 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:35.706975 (+ 1513us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:35.708518 (+ 1543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.708526 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.708528 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.708819 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.709152 (+ 333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.709153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.709153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.709305 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.709598 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:35.709605 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:35.709776 (+ 171us) client_negotiation.cc:769] Sending connection context
0809 02:14:35.709904 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:14:35.710067 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":132,"threads_started":1}
I20260809 02:14:35.710775 10842 heartbeater.cc:347] Connected to a master server at 127.9.40.126:32899
I20260809 02:14:35.710878 10842 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:35.711119 10842 heartbeater.cc:511] Master 127.9.40.126:32899 requested a full tablet report, sending...
I20260809 02:14:35.712030 10615 ts_manager.cc:194] Registered new tserver with Master: f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249)
I20260809 02:14:35.713033 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:51933
I20260809 02:14:35.718431 10849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.711878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.65:53005)
0809 02:14:35.712028 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:35.712038 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:35.712115 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:35.712243 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:35.712244 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:35.712430 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:35.712480 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.712481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.713370 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.713371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.714265 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.714268 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.714725 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.714726 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.714815 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.715223 (+ 408us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:35.715228 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:35.715679 (+ 451us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:35.717337 (+ 1658us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.717338 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.717338 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.717527 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.717751 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.717752 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.717753 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.717815 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.718073 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:35.718074 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:35.718127 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:14:35.718217 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:14:35.718331 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":95,"threads_started":1}
I20260809 02:14:35.718427 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.711834 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53005 (local address 127.9.40.126:32899)
0809 02:14:35.711875 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:35.711877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:35.712103 (+ 226us) server_negotiation.cc:407] Connection header received
0809 02:14:35.712171 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:35.712172 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:35.712182 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:35.712238 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:35.712554 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.713249 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.714357 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.714639 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.714701 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.715773 (+ 1072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:35.715774 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:35.715775 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:35.715781 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:35.717229 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.717619 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.717620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.717621 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.717682 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.717888 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.717889 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.717889 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.718001 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:35.718079 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:35.718244 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:14:35.718330 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":17}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:35.740353 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:32899
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:35.823719 10851 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:35.823983 10851 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:35.824028 10851 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:35.825892 10851 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:35.825963 10851 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:35.825999 10851 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:35.826035 10851 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:35.827518 10851 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:32899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10851
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:35.827798 10851 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:35.828051 10851 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:35.833165 10860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.833180 10851 server_base.cc:1034] running on GCE node
W20260809 02:14:35.833141 10858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.833161 10862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.833606 10851 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:35.833819 10851 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:35.835006 10851 hybrid_clock.cc:648] HybridClock initialized: now 1786241675834912 us; error 106 us; skew 500 ppm
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:35.837785 10851 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:35.838266 10851 webserver.cc:460] Webserver started at http://127.9.40.66:37299/ using document root <none> and password file <none>
I20260809 02:14:35.838534 10851 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:35.838606 10851 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:35.838759 10851 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:35.839857 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "d087f1f871dbb9c3eb74401c928226ba"
server_key_iv: "a8e2d90c26fbbe2a04dda94efce49822"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.840267 10851 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "d087f1f871dbb9c3eb74401c928226ba"
server_key_iv: "a8e2d90c26fbbe2a04dda94efce49822"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.841722 10851 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:35.842474 10870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.842650 10851 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.842764 10851 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "d087f1f871dbb9c3eb74401c928226ba"
server_key_iv: "a8e2d90c26fbbe2a04dda94efce49822"
server_key_version: "encryptionkey@0"
I20260809 02:14:35.842844 10851 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:35.855155 10851 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:35.857682 10851 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:35.857858 10851 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:35.858506 10851 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:35.858577 10851 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.858616 10851 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:35.858644 10851 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:35.865988 10851 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46621
I20260809 02:14:35.866154 10994 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46621 every 8 connection(s)
I20260809 02:14:35.866449 10851 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:35.869593 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.859012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35201 (local address 127.9.40.126:32899)
0809 02:14:35.859056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:35.859058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:35.859307 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:14:35.859863 (+ 556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:35.859865 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:35.859879 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:35.859931 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:35.861163 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.861925 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.863484 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.863801 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.863871 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.867069 (+ 3198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:35.867071 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:35.867071 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:35.867078 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:35.868215 (+ 1137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.868674 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.868675 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.868675 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.868776 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.869028 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.869028 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.869029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.869153 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:35.869216 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:35.869456 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:14:35.869529 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:35.869737 10881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.859012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.66:35201)
0809 02:14:35.859178 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:35.859215 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:35.859741 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:35.860070 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:35.860073 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:35.860470 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:35.860778 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.860818 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.862078 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.862079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.863385 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.863390 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.864232 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.864233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.864401 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.865001 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:35.865013 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:35.866971 (+ 1958us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:35.868308 (+ 1337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.868316 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.868318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.868604 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.868848 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.868848 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.868849 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.868961 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.869219 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:35.869224 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:35.869342 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 02:14:35.869426 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:35.869550 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":92,"threads_started":1}
I20260809 02:14:35.870232 10995 heartbeater.cc:347] Connected to a master server at 127.9.40.126:32899
I20260809 02:14:35.870322 10995 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:35.870518 10995 heartbeater.cc:511] Master 127.9.40.126:32899 requested a full tablet report, sending...
I20260809 02:14:35.870921 10615 ts_manager.cc:194] Registered new tserver with Master: d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621)
I20260809 02:14:35.871346 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35201
I20260809 02:14:35.876132 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 10851
I20260809 02:14:35.876261 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:35.876453 9377 external_mini_cluster.cc:1428] Setting key faaddbd25bf193e9c15e6a36b8a80c90
I20260809 02:14:35.876889 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.871422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:48825 (local address 127.9.40.126:32899)
0809 02:14:35.871466 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:35.871468 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:35.871577 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:14:35.871648 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:35.871649 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:35.871659 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:35.871711 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:35.872042 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.872548 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.873299 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.873496 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.873553 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.874466 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:35.874467 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:35.874467 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:35.874489 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:35.875638 (+ 1149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.875951 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.875952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.875953 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.876015 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:35.876198 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:35.876199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:35.876200 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:35.876411 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:35.876509 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:35.876673 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:14:35.876791 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:35.876899 11002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:35.871367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.66:48825)
0809 02:14:35.871498 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:35.871508 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:35.871588 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:35.871738 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:35.871739 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:35.871895 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:35.871955 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.871956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.872642 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.872643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.873225 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:35.873227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:35.873567 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:35.873568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:35.873622 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:35.874112 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:35.874117 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:35.874387 (+ 270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:35.875712 (+ 1325us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.875712 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.875713 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.875875 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.876082 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:35.876082 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:35.876083 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:35.876134 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:35.876482 (+ 348us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:35.876483 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:35.876552 (+ 69us) client_negotiation.cc:769] Sending connection context
0809 02:14:35.876624 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:14:35.876783 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":107,"thread_start_us":67,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:35.903941 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:32899
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42609
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:35.987625 11006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:35.987838 11006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:35.987862 11006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:35.989194 11006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:35.989243 11006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:35.989266 11006 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:35.989284 11006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:35.990756 11006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42609
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:32899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:35.991019 11006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:35.991266 11006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:35.996439 11018 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.996439 11016 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:35.996419 11015 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:35.996624 11006 server_base.cc:1034] running on GCE node
I20260809 02:14:35.996937 11006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:35.997162 11006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:35.998310 11006 hybrid_clock.cc:648] HybridClock initialized: now 1786241675998302 us; error 30 us; skew 500 ppm
Aug 09 02:14:35 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:36.001022 11006 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:36.001507 11006 webserver.cc:460] Webserver started at http://127.9.40.67:36177/ using document root <none> and password file <none>
I20260809 02:14:36.001777 11006 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:36.001847 11006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:36.002007 11006 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:36.003083 11006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "1920b7bab7a147bf9034cf413da468b9"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "04337efe2cb3461f97ff4898d58110fb"
server_key_iv: "1963ee61f7141b7100dea74e9a2c5d53"
server_key_version: "encryptionkey@0"
I20260809 02:14:36.003466 11006 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "1920b7bab7a147bf9034cf413da468b9"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "04337efe2cb3461f97ff4898d58110fb"
server_key_iv: "1963ee61f7141b7100dea74e9a2c5d53"
server_key_version: "encryptionkey@0"
I20260809 02:14:36.004930 11006 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 02:14:36.005738 11026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:36.005916 11006 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:36.006042 11006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "1920b7bab7a147bf9034cf413da468b9"
format_stamp: "Formatted at 2026-08-09 02:14:35 on dist-test-slave-qg60"
server_key: "04337efe2cb3461f97ff4898d58110fb"
server_key_iv: "1963ee61f7141b7100dea74e9a2c5d53"
server_key_version: "encryptionkey@0"
I20260809 02:14:36.006130 11006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:36.022051 11006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:36.024545 11006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:36.024775 11006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:36.025326 11006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:36.025393 11006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:36.025441 11006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:36.025470 11006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:36.032331 11006 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44125
I20260809 02:14:36.032398 11152 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44125 every 8 connection(s)
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:36.032815 11006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:36.035874 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.025882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:38337 (local address 127.9.40.126:32899)
0809 02:14:36.025952 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.025955 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.026182 (+ 227us) server_negotiation.cc:407] Connection header received
0809 02:14:36.026722 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.026724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.026737 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.026791 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.027701 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.028577 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.030272 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.030588 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.030682 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.033330 (+ 2648us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.033332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.033332 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.033340 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.034566 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.034937 (+ 371us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.034938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.034938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.034993 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.035259 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.035261 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.035261 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.035387 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.035488 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.035736 (+ 248us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.035810 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:14:36.036010 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.025885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.67:38337)
0809 02:14:36.026061 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.026101 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.026604 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.026900 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.026905 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.027287 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:36.027582 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.027588 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.028723 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.028724 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.030169 (+ 1445us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.030172 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.030686 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.030687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.030794 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.031372 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:36.031382 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:36.033242 (+ 1860us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:36.034647 (+ 1405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.034653 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.034655 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.034866 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.035061 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.035062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.035062 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.035169 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.035495 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:36.035499 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:36.035610 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.035702 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.035819 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":107,"threads_started":1}
I20260809 02:14:36.036521 11153 heartbeater.cc:347] Connected to a master server at 127.9.40.126:32899
I20260809 02:14:36.036633 11153 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:36.036839 11153 heartbeater.cc:511] Master 127.9.40.126:32899 requested a full tablet report, sending...
I20260809 02:14:36.037221 10615 ts_manager.cc:194] Registered new tserver with Master: 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67:44125)
I20260809 02:14:36.037736 10615 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:38337
I20260809 02:14:36.039382 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 11006
I20260809 02:14:36.039500 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:36.039652 9377 external_mini_cluster.cc:1428] Setting key 2e1954d406996c35bdd562b2ffab3ad1
I20260809 02:14:36.040771 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:14:36.043267 11160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.037563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.9.40.67:47917)
0809 02:14:36.037698 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.037708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.037774 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.037904 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.037905 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.038079 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:36.038125 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.038127 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.038886 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.038886 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.039361 (+ 475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.039362 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.039875 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.039876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.039932 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.040290 (+ 358us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:36.040294 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:36.040625 (+ 331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:36.042193 (+ 1568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.042194 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.042194 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.042369 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.042582 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.042583 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.042584 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.042656 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.042964 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:36.042966 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:36.043025 (+ 59us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.043105 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.043192 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":84,"threads_started":1}
I20260809 02:14:36.043355 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.037655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:47917 (local address 127.9.40.126:32899)
0809 02:14:36.037694 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.037696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.037763 (+ 67us) server_negotiation.cc:407] Connection header received
0809 02:14:36.037827 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.037828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.037838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.037899 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.038185 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.038798 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.039441 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.039723 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.039795 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.040753 (+ 958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.040755 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.040755 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.040762 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.042071 (+ 1309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.042447 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.042448 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.042449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.042513 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.042723 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.042725 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.042725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.042858 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.042939 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.043153 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.043240 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[10565](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241676, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241676, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:36.065577 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.058751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56122 (local address 127.9.40.126:32899)
0809 02:14:36.058811 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.058813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.058831 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:14:36.058857 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.058858 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.058868 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.058923 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.059628 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.060119 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.060905 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.061080 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.061143 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.063433 (+ 2290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.063434 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.063434 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.063440 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.064610 (+ 1170us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.064979 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.064980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.064980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.065033 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.065195 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.065196 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.065196 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.065289 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.065361 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.065449 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.065508 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:14:36.067258 10615 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:36.068924 10615 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:36.075659 11176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45249 (local address 127.0.0.1:40646)
0809 02:14:36.071761 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.071805 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.071917 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.072188 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.072189 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.072190 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.072228 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.072232 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073305 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.073306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.074371 (+ 1065us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.074374 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.075334 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.075335 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.075418 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.075432 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.075504 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.075559 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":72,"threads_started":1}
I20260809 02:14:36.075670 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40646 (local address 127.9.40.65:45249)
0809 02:14:36.071668 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.071673 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.071882 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:14:36.072021 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.072023 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.072041 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.072095 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.072300 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073215 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.074490 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.075250 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.075334 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.075399 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.075531 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.075575 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260809 02:14:36.075940 11172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46621 (local address 127.0.0.1:59210)
0809 02:14:36.071951 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.071962 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.072005 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.072479 (+ 474us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.072479 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.072482 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.072522 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.072523 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073907 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.073908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.074806 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.074808 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.075706 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.075707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.075769 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.075773 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.075830 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.075867 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":88,"threads_started":1}
I20260809 02:14:36.076225 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59210 (local address 127.9.40.66:46621)
0809 02:14:36.072045 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.072055 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.072065 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:36.072104 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.072106 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.072121 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.072418 (+ 297us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.072615 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073797 (+ 1182us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.074892 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.075544 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.075616 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.075668 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.075858 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.075907 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":92,"threads_started":1}
I20260809 02:14:36.076457 11174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44125 (local address 127.0.0.1:57878)
0809 02:14:36.071765 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.071808 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.071874 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.072150 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.072153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.072156 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.072229 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.072231 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073942 (+ 1711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.073943 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.075064 (+ 1121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.075067 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.076232 (+ 1165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.076233 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.076311 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.076316 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.076351 (+ 35us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.076384 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"mutex_wait_us":2,"thread_start_us":85,"threads_started":1}
I20260809 02:14:36.076599 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 (DEFAULT_TABLE table=test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.076959 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.071676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57878 (local address 127.9.40.67:44125)
0809 02:14:36.071883 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.071888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.071898 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:36.071991 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.071995 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.072013 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.072189 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.072805 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.073871 (+ 1066us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.075157 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.076123 (+ 966us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.076712 (+ 589us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.076811 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.076838 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.076884 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":95,"threads_started":1}
I20260809 02:14:36.076880 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 (DEFAULT_TABLE table=test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.077041 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc04f00f9e74eb8b784ceab4acfda93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.077209 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc04f00f9e74eb8b784ceab4acfda93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.077870 11079 tablet_service.cc:1467] Processing CreateTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 (DEFAULT_TABLE table=test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.078172 11079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3fc04f00f9e74eb8b784ceab4acfda93. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.081460 11183 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Bootstrap starting.
I20260809 02:14:36.081522 11182 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Bootstrap starting.
I20260809 02:14:36.081522 11180 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Bootstrap starting.
I20260809 02:14:36.082187 11182 tablet_bootstrap.cc:654] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.082188 11183 tablet_bootstrap.cc:654] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.082561 11180 tablet_bootstrap.cc:654] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.082566 11182 log.cc:824] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:36.082599 11183 log.cc:824] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:36.083050 11180 log.cc:824] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:36.083590 11183 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: No bootstrap required, opened a new log
I20260809 02:14:36.083676 11183 ts_tablet_manager.cc:1397] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:36.083881 11180 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: No bootstrap required, opened a new log
I20260809 02:14:36.083956 11180 ts_tablet_manager.cc:1397] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:14:36.085342 11182 tablet_bootstrap.cc:492] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: No bootstrap required, opened a new log
I20260809 02:14:36.085438 11182 ts_tablet_manager.cc:1397] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 02:14:36.086089 11183 raft_consensus.cc:348] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.086313 11183 raft_consensus.cc:374] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.086356 11183 raft_consensus.cc:729] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58fa7f32a314d8ea4f75e0d97ea4c58, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.086246 11180 raft_consensus.cc:348] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.086408 11180 raft_consensus.cc:374] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.086442 11180 raft_consensus.cc:729] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1920b7bab7a147bf9034cf413da468b9, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.086493 11183 consensus_queue.cc:260] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.086592 11180 consensus_queue.cc:260] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.086838 11183 ts_tablet_manager.cc:1428] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260809 02:14:36.086881 11180 ts_tablet_manager.cc:1428] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:36.086992 10842 heartbeater.cc:503] Master 127.9.40.126:32899 was elected leader, sending a full tablet report...
I20260809 02:14:36.087035 11153 heartbeater.cc:503] Master 127.9.40.126:32899 was elected leader, sending a full tablet report...
I20260809 02:14:36.087558 11182 raft_consensus.cc:348] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.087711 11182 raft_consensus.cc:374] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.087759 11182 raft_consensus.cc:729] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f3d1fd01694719bc0ffc2c78e00f07, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.087879 11182 consensus_queue.cc:260] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.088214 10995 heartbeater.cc:503] Master 127.9.40.126:32899 was elected leader, sending a full tablet report...
I20260809 02:14:36.088938 11182 ts_tablet_manager.cc:1428] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260809 02:14:36.117066 10997 tablet.cc:2369] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:36.121217 11188 raft_consensus.cc:482] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:36.121389 11188 raft_consensus.cc:504] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.121806 11188 leader_election.cc:288] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249), d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621)
I20260809 02:14:36.125005 11035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.122034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45249 (local address 127.9.40.67:47445)
0809 02:14:36.122107 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.122121 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.122168 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.122408 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.122409 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.122411 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.122481 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.122483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.123320 (+ 837us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.123321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.124124 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.124126 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.124699 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.124814 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.124817 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.124878 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.124907 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:14:36.125005 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.122124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:47445 (local address 127.9.40.65:45249)
0809 02:14:36.122224 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.122226 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.122235 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:36.122286 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.122287 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.122301 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.122392 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.122582 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.123233 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124228 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.124616 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124671 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.124680 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.124885 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.124929 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260809 02:14:36.125116 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.122228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46959 (local address 127.9.40.66:46621)
0809 02:14:36.122306 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.122308 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.122551 (+ 243us) server_negotiation.cc:407] Connection header received
0809 02:14:36.122623 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.122624 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.122650 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.122689 (+ 39us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.122880 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.123372 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124322 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.124705 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124807 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.124817 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.124992 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.125042 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:14:36.125178 11191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.122039 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46621 (local address 127.9.40.67:46959)
0809 02:14:36.122457 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.122468 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.122532 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.122718 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.122718 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.122720 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.122802 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.122804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.123466 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.123467 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.124221 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.124224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.124806 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.124807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.124901 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.124905 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.124964 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.125002 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":185,"thread_start_us":146,"threads_started":1}
I20260809 02:14:36.125422 10792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc04f00f9e74eb8b784ceab4acfda93" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" is_pre_election: true
I20260809 02:14:36.125597 10792 raft_consensus.cc:2454] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 0.
I20260809 02:14:36.125700 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc04f00f9e74eb8b784ceab4acfda93" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" is_pre_election: true
I20260809 02:14:36.125856 10935 raft_consensus.cc:2454] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 0.
I20260809 02:14:36.125902 11029 leader_election.cc:302] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1920b7bab7a147bf9034cf413da468b9, f58fa7f32a314d8ea4f75e0d97ea4c58; no voters:
I20260809 02:14:36.126044 11188 raft_consensus.cc:2781] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:36.126101 11188 raft_consensus.cc:482] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:36.126118 11188 raft_consensus.cc:3037] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.126876 11188 raft_consensus.cc:504] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.127025 11188 leader_election.cc:288] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 election: Requested vote from peers f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249), d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621)
I20260809 02:14:36.127192 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc04f00f9e74eb8b784ceab4acfda93" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
I20260809 02:14:36.127210 10792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3fc04f00f9e74eb8b784ceab4acfda93" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58"
I20260809 02:14:36.127282 10792 raft_consensus.cc:3037] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.127282 10935 raft_consensus.cc:3037] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.128214 10792 raft_consensus.cc:2454] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 1.
I20260809 02:14:36.128214 10935 raft_consensus.cc:2454] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 1.
I20260809 02:14:36.128402 11029 leader_election.cc:302] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1920b7bab7a147bf9034cf413da468b9, f58fa7f32a314d8ea4f75e0d97ea4c58; no voters:
I20260809 02:14:36.128516 11188 raft_consensus.cc:2781] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:36.128717 11188 raft_consensus.cc:686] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1920b7bab7a147bf9034cf413da468b9, State: Running, Role: LEADER
I20260809 02:14:36.128880 11188 consensus_queue.cc:237] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } }
I20260809 02:14:36.129778 10616 catalog_manager.cc:5495] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } health_report { overall_health: UNKNOWN } } }
I20260809 02:14:36.135871 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.133571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57882 (local address 127.9.40.67:44125)
0809 02:14:36.133610 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.133612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.133661 (+ 49us) server_negotiation.cc:407] Connection header received
0809 02:14:36.133735 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.133736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.133750 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.133808 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:36.133925 (+ 117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.134376 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.135120 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.135303 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.135354 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.135594 (+ 240us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.135681 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.135738 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.135789 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.135805 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:36.138080 10792 raft_consensus.cc:1261] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Refusing update from remote peer 1920b7bab7a147bf9034cf413da468b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.138151 10935 raft_consensus.cc:1261] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Refusing update from remote peer 1920b7bab7a147bf9034cf413da468b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.138446 11192 consensus_queue.cc:1035] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.138587 11188 consensus_queue.cc:1035] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.141033 11195 mvcc.cc:205] Tried to move back new op lower bound from 7316445905459105792 to 7316445905424728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316445905459105792}]
I20260809 02:14:36.142664 11197 mvcc.cc:205] Tried to move back new op lower bound from 7316445905459105792 to 7316445905424728064. Current Snapshot: MvccSnapshot[applied={T|T < 7316445905459105792 or (T in {7316445905459105792})}]
I20260809 02:14:36.144783 10616 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:36.144990 10616 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:36.145865 10616 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Sending DeleteTablet for 3 replicas of tablet 3fc04f00f9e74eb8b784ceab4acfda93
I20260809 02:14:36.146359 11079 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:57878
I20260809 02:14:36.146378 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40646
I20260809 02:14:36.146661 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 3fc04f00f9e74eb8b784ceab4acfda93 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260809 02:14:36.146935 11206 tablet_replica.cc:330] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: stopping tablet replica
I20260809 02:14:36.146917 10616 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:14:36.147049 11206 raft_consensus.cc:2229] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 1 LEADER]: Raft consensus shutting down.
W20260809 02:14:36.147096 10616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:36.147151 11206 raft_consensus.cc:2258] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.147307 11208 tablet_replica.cc:330] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: stopping tablet replica
I20260809 02:14:36.147418 11208 raft_consensus.cc:2229] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:36.147532 11208 raft_consensus.cc:2258] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.147722 11209 tablet_replica.cc:330] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: stopping tablet replica
I20260809 02:14:36.147908 11209 raft_consensus.cc:2229] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:36.147924 11206 ts_tablet_manager.cc:1905] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.148370 11208 ts_tablet_manager.cc:1905] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.148458 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2ee7b816d249328196a8483f83d525 (DEFAULT_TABLE table=test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.148593 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2ee7b816d249328196a8483f83d525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.148578 11079 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2ee7b816d249328196a8483f83d525 (DEFAULT_TABLE table=test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.148671 11079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2ee7b816d249328196a8483f83d525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.149801 11206 ts_tablet_manager.cc:1918] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:36.149873 11206 log.cc:1196] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/3fc04f00f9e74eb8b784ceab4acfda93
I20260809 02:14:36.150005 11183 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Bootstrap starting.
I20260809 02:14:36.150007 11180 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Bootstrap starting.
I20260809 02:14:36.150137 11206 ts_tablet_manager.cc:1939] T 3fc04f00f9e74eb8b784ceab4acfda93 P 1920b7bab7a147bf9034cf413da468b9: Deleting consensus metadata
I20260809 02:14:36.150440 11183 tablet_bootstrap.cc:654] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.150449 11180 tablet_bootstrap.cc:654] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.150548 11208 ts_tablet_manager.cc:1918] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:36.150611 11208 log.cc:1196] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/3fc04f00f9e74eb8b784ceab4acfda93
I20260809 02:14:36.150857 11208 ts_tablet_manager.cc:1939] T 3fc04f00f9e74eb8b784ceab4acfda93 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting consensus metadata
I20260809 02:14:36.151154 10598 catalog_manager.cc:4841] TS 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67:44125): tablet 3fc04f00f9e74eb8b784ceab4acfda93 (table test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]) successfully deleted
I20260809 02:14:36.151223 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 3f2ee7b816d249328196a8483f83d525 (DEFAULT_TABLE table=test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:36.151317 11180 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: No bootstrap required, opened a new log
I20260809 02:14:36.151358 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 3f2ee7b816d249328196a8483f83d525. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.151360 11180 ts_tablet_manager.cc:1397] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:36.151510 11180 raft_consensus.cc:348] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.151572 11180 raft_consensus.cc:374] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.151593 11180 raft_consensus.cc:729] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1920b7bab7a147bf9034cf413da468b9, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.151635 11180 consensus_queue.cc:260] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.151714 11180 ts_tablet_manager.cc:1428] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:36.152210 11183 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: No bootstrap required, opened a new log
I20260809 02:14:36.152252 11183 ts_tablet_manager.cc:1397] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:14:36.152398 11183 raft_consensus.cc:348] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.152460 11183 raft_consensus.cc:374] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.152482 11183 raft_consensus.cc:729] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58fa7f32a314d8ea4f75e0d97ea4c58, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.152531 11183 consensus_queue.cc:260] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.152611 11183 ts_tablet_manager.cc:1428] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:36.152823 11182 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Bootstrap starting.
I20260809 02:14:36.153257 11182 tablet_bootstrap.cc:654] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.153964 11182 tablet_bootstrap.cc:492] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: No bootstrap required, opened a new log
I20260809 02:14:36.154011 10599 catalog_manager.cc:4841] TS f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249): tablet 3fc04f00f9e74eb8b784ceab4acfda93 (table test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]) successfully deleted
I20260809 02:14:36.154042 11182 ts_tablet_manager.cc:1397] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:36.154204 11182 raft_consensus.cc:348] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.154289 11182 raft_consensus.cc:374] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.154316 11182 raft_consensus.cc:729] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f3d1fd01694719bc0ffc2c78e00f07, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.154361 11182 consensus_queue.cc:260] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.154456 11182 ts_tablet_manager.cc:1428] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:36.154484 11209 raft_consensus.cc:2258] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.155010 11209 ts_tablet_manager.cc:1905] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.156890 11209 ts_tablet_manager.cc:1918] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:36.156973 11209 log.cc:1196] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/3fc04f00f9e74eb8b784ceab4acfda93
I20260809 02:14:36.157301 11209 ts_tablet_manager.cc:1939] T 3fc04f00f9e74eb8b784ceab4acfda93 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting consensus metadata
I20260809 02:14:36.157734 10598 catalog_manager.cc:4841] TS d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621): tablet 3fc04f00f9e74eb8b784ceab4acfda93 (table test-table [id=276a2b9e70f147b1aba2ac41c57bee4b]) successfully deleted
W20260809 02:14:36.207770 10843 tablet.cc:2369] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:14:36.283447 11154 tablet.cc:2369] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:36.321403 11187 raft_consensus.cc:482] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:36.321512 11187 raft_consensus.cc:504] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.321909 11187 leader_election.cc:288] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621), 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67:44125)
I20260809 02:14:36.324963 11178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.322249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:59627 (local address 127.9.40.66:46621)
0809 02:14:36.322315 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.322332 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.322348 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:14:36.322417 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.322418 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.322429 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.322516 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.322699 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.323184 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324178 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.324531 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324616 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.324627 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.324845 (+ 218us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.324884 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:14:36.325001 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.322432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:34485 (local address 127.9.40.67:44125)
0809 02:14:36.322524 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.322526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.322537 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:36.322601 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.322603 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.322621 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.322687 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:36.322852 (+ 165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.323311 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324223 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.324597 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324683 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.324694 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.324891 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.324930 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:14:36.325014 11213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.322138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:46621 (local address 127.9.40.65:59627)
0809 02:14:36.322276 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.322289 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.322349 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.322526 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.322527 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.322530 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.322615 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.322616 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.323274 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.323275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.324059 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.324062 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.324621 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324621 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.324706 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.324710 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.324834 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.324881 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":82,"threads_started":1}
I20260809 02:14:36.325026 11215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.322335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44125 (local address 127.9.40.65:34485)
0809 02:14:36.322470 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.322480 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.322529 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.322693 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.322694 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.322696 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:36.322776 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.322778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.323400 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.323401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.324123 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.324125 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.324687 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.324688 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.324813 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.324816 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.324883 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.324926 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":41,"threads_started":1}
I20260809 02:14:36.325521 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2ee7b816d249328196a8483f83d525" candidate_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" is_pre_election: true
I20260809 02:14:36.325512 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2ee7b816d249328196a8483f83d525" candidate_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1920b7bab7a147bf9034cf413da468b9" is_pre_election: true
I20260809 02:14:36.325640 11103 raft_consensus.cc:2454] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58fa7f32a314d8ea4f75e0d97ea4c58 in term 0.
I20260809 02:14:36.325655 10935 raft_consensus.cc:2454] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f58fa7f32a314d8ea4f75e0d97ea4c58 in term 0.
I20260809 02:14:36.325855 10716 leader_election.cc:302] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4f3d1fd01694719bc0ffc2c78e00f07, f58fa7f32a314d8ea4f75e0d97ea4c58; no voters:
I20260809 02:14:36.326022 11187 raft_consensus.cc:2781] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:36.326079 11187 raft_consensus.cc:482] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:36.326098 11187 raft_consensus.cc:3037] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.326823 11187 raft_consensus.cc:504] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.326963 11187 leader_election.cc:288] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [CANDIDATE]: Term 1 election: Requested vote from peers d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621), 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67:44125)
I20260809 02:14:36.327147 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2ee7b816d249328196a8483f83d525" candidate_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
I20260809 02:14:36.327174 11103 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "3f2ee7b816d249328196a8483f83d525" candidate_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1920b7bab7a147bf9034cf413da468b9"
I20260809 02:14:36.327224 10935 raft_consensus.cc:3037] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.327225 11103 raft_consensus.cc:3037] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.327876 11103 raft_consensus.cc:2454] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58fa7f32a314d8ea4f75e0d97ea4c58 in term 1.
I20260809 02:14:36.327909 10935 raft_consensus.cc:2454] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f58fa7f32a314d8ea4f75e0d97ea4c58 in term 1.
I20260809 02:14:36.328100 10716 leader_election.cc:302] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d4f3d1fd01694719bc0ffc2c78e00f07, f58fa7f32a314d8ea4f75e0d97ea4c58; no voters:
I20260809 02:14:36.328207 11187 raft_consensus.cc:2781] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:36.328342 11187 raft_consensus.cc:686] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 LEADER]: Becoming Leader. State: Replica: f58fa7f32a314d8ea4f75e0d97ea4c58, State: Running, Role: LEADER
I20260809 02:14:36.328416 11187 consensus_queue.cc:237] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.329237 10616 catalog_manager.cc:5495] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 reported cstate change: term changed from 0 to 1, leader changed from <none> to f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241675, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:36.368628 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.361560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:32899 (local address 127.0.0.1:56124)
0809 02:14:36.361718 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.361729 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.361801 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.361937 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.361938 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.362274 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:36.362325 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.362327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.363031 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.363032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.363561 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.363563 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.363913 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.363914 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.364039 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.364655 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:36.364660 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:36.366200 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:36.367608 (+ 1408us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.367615 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.367616 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.367826 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.368026 (+ 200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:36.368028 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:36.368028 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:36.368097 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:36.368316 (+ 219us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:36.368322 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:36.368365 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.368418 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.368514 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":68,"threads_started":1}
I20260809 02:14:36.368647 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.361628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56124 (local address 127.9.40.126:32899)
0809 02:14:36.361731 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.361733 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.361835 (+ 102us) server_negotiation.cc:407] Connection header received
0809 02:14:36.361859 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.361860 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.361870 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.361910 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.362401 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.362892 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.363648 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.363842 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.363894 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.366289 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.366290 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.366290 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.366296 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.367522 (+ 1226us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.367894 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.367895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.367895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.367945 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.368162 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.368162 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.368162 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.368260 (+ 98us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.368310 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.368449 (+ 139us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.368561 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":65}
I20260809 02:14:36.370921 10616 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:56124:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:14:36.371239 10616 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:36.372866 11079 tablet_service.cc:1467] Processing CreateTablet for tablet 120128c01bbd4e97b067007eaa298ec1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7937617057bc4be093f8cbbce512aaa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:36.372839 10918 tablet_service.cc:1467] Processing CreateTablet for tablet 120128c01bbd4e97b067007eaa298ec1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7937617057bc4be093f8cbbce512aaa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:36.372803 10772 tablet_service.cc:1467] Processing CreateTablet for tablet 120128c01bbd4e97b067007eaa298ec1 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=7937617057bc4be093f8cbbce512aaa9]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:36.373011 11079 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120128c01bbd4e97b067007eaa298ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.373021 10918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120128c01bbd4e97b067007eaa298ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.373020 10772 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 120128c01bbd4e97b067007eaa298ec1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:36.374281 11180 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: Bootstrap starting.
I20260809 02:14:36.374517 11182 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: Bootstrap starting.
I20260809 02:14:36.374531 11183 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: Bootstrap starting.
I20260809 02:14:36.374667 11180 tablet_bootstrap.cc:654] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.374958 11182 tablet_bootstrap.cc:654] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.375026 11183 tablet_bootstrap.cc:654] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:36.375356 11180 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: No bootstrap required, opened a new log
I20260809 02:14:36.375414 11180 ts_tablet_manager.cc:1397] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:36.375552 11180 raft_consensus.cc:348] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.375630 11180 raft_consensus.cc:374] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.375653 11180 raft_consensus.cc:729] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1920b7bab7a147bf9034cf413da468b9, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.375703 11180 consensus_queue.cc:260] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.375778 11183 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: No bootstrap required, opened a new log
I20260809 02:14:36.375828 11183 ts_tablet_manager.cc:1397] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:36.375823 11180 ts_tablet_manager.cc:1428] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:36.375980 11183 raft_consensus.cc:348] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.376055 11183 raft_consensus.cc:374] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.376080 11183 raft_consensus.cc:729] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f58fa7f32a314d8ea4f75e0d97ea4c58, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.376128 11183 consensus_queue.cc:260] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.376149 11182 tablet_bootstrap.cc:492] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: No bootstrap required, opened a new log
I20260809 02:14:36.376194 11182 ts_tablet_manager.cc:1397] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:36.376219 11183 ts_tablet_manager.cc:1428] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:36.376286 11187 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.376313 11203 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.376382 11187 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.376358 11182 raft_consensus.cc:348] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.376410 11203 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.376426 11182 raft_consensus.cc:374] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:36.376449 11182 raft_consensus.cc:729] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f3d1fd01694719bc0ffc2c78e00f07, State: Initialized, Role: FOLLOWER
I20260809 02:14:36.376488 11182 consensus_queue.cc:260] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.376592 11182 ts_tablet_manager.cc:1428] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:36.376637 11190 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.376703 11190 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.495519 11203 raft_consensus.cc:482] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:36.495673 11203 raft_consensus.cc:504] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.495888 11203 leader_election.cc:288] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249), d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621)
I20260809 02:14:36.496122 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120128c01bbd4e97b067007eaa298ec1" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" is_pre_election: true
I20260809 02:14:36.496212 10935 raft_consensus.cc:2454] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 0.
I20260809 02:14:36.496138 10792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120128c01bbd4e97b067007eaa298ec1" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" is_pre_election: true
I20260809 02:14:36.496276 10792 raft_consensus.cc:2454] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 0.
I20260809 02:14:36.496385 11028 leader_election.cc:302] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1920b7bab7a147bf9034cf413da468b9, d4f3d1fd01694719bc0ffc2c78e00f07; no voters:
I20260809 02:14:36.496526 11203 raft_consensus.cc:2781] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:36.496574 11203 raft_consensus.cc:482] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:36.496588 11203 raft_consensus.cc:3037] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.497375 11203 raft_consensus.cc:504] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.497520 11203 leader_election.cc:288] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 election: Requested vote from peers f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249), d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621)
I20260809 02:14:36.497704 10935 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120128c01bbd4e97b067007eaa298ec1" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07"
I20260809 02:14:36.497746 10792 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "120128c01bbd4e97b067007eaa298ec1" candidate_uuid: "1920b7bab7a147bf9034cf413da468b9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58"
I20260809 02:14:36.497797 10935 raft_consensus.cc:3037] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.497813 10792 raft_consensus.cc:3037] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:36.498556 10935 raft_consensus.cc:2454] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 1.
I20260809 02:14:36.498556 10792 raft_consensus.cc:2454] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1920b7bab7a147bf9034cf413da468b9 in term 1.
I20260809 02:14:36.498756 11028 leader_election.cc:302] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1920b7bab7a147bf9034cf413da468b9, d4f3d1fd01694719bc0ffc2c78e00f07; no voters:
I20260809 02:14:36.498896 11203 raft_consensus.cc:2781] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:36.498982 11203 raft_consensus.cc:686] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [term 1 LEADER]: Becoming Leader. State: Replica: 1920b7bab7a147bf9034cf413da468b9, State: Running, Role: LEADER
I20260809 02:14:36.499058 11203 consensus_queue.cc:237] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } }
I20260809 02:14:36.499332 11188 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: TxnStatusTablet state changed. Reason: New leader 1920b7bab7a147bf9034cf413da468b9. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.499429 11188 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:36.499657 11229 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:14:36.499814 11229 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:14:36.499722 10614 catalog_manager.cc:5495] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } health_report { overall_health: HEALTHY } } }
I20260809 02:14:36.524549 11223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.521922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44125 (local address 127.0.0.1:57898)
0809 02:14:36.522002 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.522014 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.522063 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.522216 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.522217 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.522218 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:36.522261 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.522262 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.523131 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.523132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.523690 (+ 558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.523692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.524154 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.524155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.524199 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.524215 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.524392 (+ 177us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.524399 (+ 7us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.524435 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.524471 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:14:36.524542 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.522009 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57898 (local address 127.9.40.67:44125)
0809 02:14:36.522057 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.522059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.522069 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:36.522129 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.522129 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.522140 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.522209 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:36.522304 (+ 95us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.523051 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.523778 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.524076 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.524148 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.524297 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.524331 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.524384 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.524443 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.524462 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:36.525745 10935 raft_consensus.cc:1261] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Refusing update from remote peer 1920b7bab7a147bf9034cf413da468b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.525758 10792 raft_consensus.cc:1261] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Refusing update from remote peer 1920b7bab7a147bf9034cf413da468b9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.525987 11188 consensus_queue.cc:1035] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.526095 11203 consensus_queue.cc:1035] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.526706 11187 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: TxnStatusTablet state changed. Reason: New leader 1920b7bab7a147bf9034cf413da468b9. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.526811 11187 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.527307 11190 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: TxnStatusTablet state changed. Reason: New leader 1920b7bab7a147bf9034cf413da468b9. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.527405 11190 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.527385 11203 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.527467 11203 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:36.528255 11203 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.528334 11203 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P 1920b7bab7a147bf9034cf413da468b9: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:36.528445 11196 mvcc.cc:205] Tried to move back new op lower bound from 7316445907047727104 to 7316445906940473344. Current Snapshot: MvccSnapshot[applied={T|T < 7316445907047727104}]
I20260809 02:14:36.528697 11187 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.528803 11187 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P f58fa7f32a314d8ea4f75e0d97ea4c58: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.529795 11190 tablet_replica.cc:439] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1920b7bab7a147bf9034cf413da468b9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f58fa7f32a314d8ea4f75e0d97ea4c58" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 45249 } } peers { permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 } } peers { permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 } } }
I20260809 02:14:36.529881 11190 tablet_replica.cc:442] T 120128c01bbd4e97b067007eaa298ec1 P d4f3d1fd01694719bc0ffc2c78e00f07: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:36.533123 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.530481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40654 (local address 127.9.40.65:45249)
0809 02:14:36.530523 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.530525 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.530558 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:14:36.530641 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.530642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.530658 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.530710 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:36.530824 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.531275 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.532264 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.532502 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.532568 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.532777 (+ 209us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.532909 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.532984 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.533030 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.533052 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:36.538641 11177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.535863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:52473 (local address 127.9.40.67:44125)
0809 02:14:36.535913 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.535914 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.536084 (+ 170us) server_negotiation.cc:407] Connection header received
0809 02:14:36.536113 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.536114 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.536124 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.536227 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:36.536315 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.536876 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.537998 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.538173 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.538230 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.538434 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.538462 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.538501 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.538576 (+ 75us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.538591 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:36.538719 11239 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.535773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44125 (local address 127.9.40.65:52473)
0809 02:14:36.535985 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.535996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.536060 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.536197 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.536198 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.536199 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:36.536258 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.536259 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.536947 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.536948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.537879 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.537882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.538271 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.538272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.538337 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.538353 (+ 16us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.538521 (+ 168us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.538522 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.538573 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.538615 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":96,"threads_started":1}
I20260809 02:14:36.541222 10935 raft_consensus.cc:1261] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Refusing update from remote peer f58fa7f32a314d8ea4f75e0d97ea4c58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.541262 11103 raft_consensus.cc:1261] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Refusing update from remote peer f58fa7f32a314d8ea4f75e0d97ea4c58: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:36.541561 11187 consensus_queue.cc:1035] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d4f3d1fd01694719bc0ffc2c78e00f07" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 46621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.541752 11216 consensus_queue.cc:1035] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1920b7bab7a147bf9034cf413da468b9" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:36.550710 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.547609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57265 (local address 127.9.40.65:45249)
0809 02:14:36.547652 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.547654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.547725 (+ 71us) server_negotiation.cc:407] Connection header received
0809 02:14:36.547852 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.547853 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.547865 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.547927 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:36.548073 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.548600 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.549919 (+ 1319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.550185 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.550223 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.550453 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.550486 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.550511 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.550624 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.550657 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:36.550738 11244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.547529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:45249 (local address 127.9.40.67:57265)
0809 02:14:36.547656 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:36.547668 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:36.547752 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:36.547932 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:36.547932 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:36.547934 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:36.548009 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.548011 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.548680 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.548681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.549823 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:36.549826 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.550281 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.550281 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:36.550348 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.550363 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:36.550548 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:36.550550 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:14:36.550609 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:14:36.550664 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"thread_start_us":66,"threads_started":1}
I20260809 02:14:36.560775 10614 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:36.560912 10614 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56122:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:36.561580 10614 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bf35738cafa44aae98853a690b765b74: Sending DeleteTablet for 3 replicas of tablet 3f2ee7b816d249328196a8483f83d525
I20260809 02:14:36.561820 10772 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2ee7b816d249328196a8483f83d525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40646
I20260809 02:14:36.561909 11208 tablet_replica.cc:330] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: stopping tablet replica
I20260809 02:14:36.561903 11079 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2ee7b816d249328196a8483f83d525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:57878
I20260809 02:14:36.561951 11208 raft_consensus.cc:2229] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:36.562036 11208 raft_consensus.cc:2258] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.562016 11206 tablet_replica.cc:330] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: stopping tablet replica
I20260809 02:14:36.562088 11206 raft_consensus.cc:2229] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:36.562129 11206 raft_consensus.cc:2258] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.562425 11206 ts_tablet_manager.cc:1905] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.562482 11208 ts_tablet_manager.cc:1905] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.563705 10918 tablet_service.cc:1514] Processing DeleteTablet for tablet 3f2ee7b816d249328196a8483f83d525 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:36 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59210
I20260809 02:14:36.563825 11209 tablet_replica.cc:330] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: stopping tablet replica
I20260809 02:14:36.563915 11209 raft_consensus.cc:2229] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:36.563967 11209 raft_consensus.cc:2258] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:36.564026 11206 ts_tablet_manager.cc:1918] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:36.564093 11206 log.cc:1196] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/3f2ee7b816d249328196a8483f83d525
I20260809 02:14:36.564327 11209 ts_tablet_manager.cc:1905] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:36.564363 11206 ts_tablet_manager.cc:1939] T 3f2ee7b816d249328196a8483f83d525 P 1920b7bab7a147bf9034cf413da468b9: Deleting consensus metadata
I20260809 02:14:36.564527 11208 ts_tablet_manager.cc:1918] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:36.564572 11208 log.cc:1196] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/3f2ee7b816d249328196a8483f83d525
I20260809 02:14:36.564707 10598 catalog_manager.cc:4841] TS 1920b7bab7a147bf9034cf413da468b9 (127.9.40.67:44125): tablet 3f2ee7b816d249328196a8483f83d525 (table test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]) successfully deleted
I20260809 02:14:36.564807 11208 ts_tablet_manager.cc:1939] T 3f2ee7b816d249328196a8483f83d525 P f58fa7f32a314d8ea4f75e0d97ea4c58: Deleting consensus metadata
I20260809 02:14:36.565135 10599 catalog_manager.cc:4841] TS f58fa7f32a314d8ea4f75e0d97ea4c58 (127.9.40.65:45249): tablet 3f2ee7b816d249328196a8483f83d525 (table test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]) successfully deleted
I20260809 02:14:36.565950 11209 ts_tablet_manager.cc:1918] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:36.566022 11209 log.cc:1196] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/3f2ee7b816d249328196a8483f83d525
I20260809 02:14:36.566252 11209 ts_tablet_manager.cc:1939] T 3f2ee7b816d249328196a8483f83d525 P d4f3d1fd01694719bc0ffc2c78e00f07: Deleting consensus metadata
I20260809 02:14:36.566761 10598 catalog_manager.cc:4841] TS d4f3d1fd01694719bc0ffc2c78e00f07 (127.9.40.66:46621): tablet 3f2ee7b816d249328196a8483f83d525 (table test-table [id=8ae0548264fc40de9054a1a4b5cff3ae]) successfully deleted
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[10565](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241676, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:36.574347 11175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.565234 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40668 (local address 127.9.40.65:45249)
0809 02:14:36.565274 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.565276 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.565570 (+ 294us) server_negotiation.cc:407] Connection header received
0809 02:14:36.565650 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.565651 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.565661 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.565717 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.566401 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.567054 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.567810 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.568015 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.568092 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.571150 (+ 3058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.571158 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.571158 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.571165 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.573039 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.573419 (+ 380us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.573422 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.573423 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.573501 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.573713 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.573714 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.573714 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.573863 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.574056 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.574094 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.574175 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:14:36.574797 10835 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:40668
I20260809 02:14:36.582530 10692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:36.575801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56134 (local address 127.9.40.126:32899)
0809 02:14:36.575864 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:36.575866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:36.576033 (+ 167us) server_negotiation.cc:407] Connection header received
0809 02:14:36.576111 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:36.576112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:36.576124 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:36.576180 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:36.576868 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.577360 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.578046 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:36.578257 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:36.578333 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:36.579518 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:36.579520 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:36.579520 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:36.579527 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:36.581146 (+ 1619us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.581609 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.581610 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.581611 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.581669 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:36.581897 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:36.581898 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:36.581898 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:36.582018 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:36.582143 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:36.582278 (+ 135us) server_negotiation.cc:299] Negotiation successful
0809 02:14:36.582356 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
W20260809 02:14:36.582839 10614 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:56134
I20260809 02:14:36.583535 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10698
I20260809 02:14:36.591280 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10851
I20260809 02:14:36.599625 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 11006
I20260809 02:14:36.606046 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 10580
2026-08-09T02:14:36Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2580 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[11255](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[11255](info): set up 2 sockets
Aug 09 02:14:36 dist-test-slave-qg60 krb5kdc[11255](info): commencing operation
krb5kdc: starting...
W20260809 02:14:37.705703 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.076s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:37 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:37.832270 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45631
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45631
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:37.920260 11269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:37.920526 11269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:37.920567 11269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:37.921766 11269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:37.921835 11269 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:14:37.921859 11269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:37.921880 11269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:37.921911 11269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:37.921941 11269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:37.923501 11269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45631
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45631
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:37.923786 11269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:37.924106 11269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:37.929632 11275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:37.929682 11278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:37.929685 11274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:37.930503 11269 server_base.cc:1034] running on GCE node
I20260809 02:14:37.930805 11269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:37.931070 11269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:37.932235 11269 hybrid_clock.cc:648] HybridClock initialized: now 1786241677932214 us; error 41 us; skew 500 ppm
Aug 09 02:14:37 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241677, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:37.934912 11269 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:37.935326 11269 webserver.cc:460] Webserver started at http://127.9.40.126:36311/ using document root <none> and password file <none>
I20260809 02:14:37.935587 11269 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:37.936295 11269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:37.936537 11269 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:37.938046 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "6d6c281d6b264ff6a00210b63af3d916"
format_stamp: "Formatted at 2026-08-09 02:14:37 on dist-test-slave-qg60"
server_key: "febcc979865ea0783a2936a1528c549e"
server_key_iv: "42927fd22138558189f216d8a5bebea2"
server_key_version: "encryptionkey@0"
I20260809 02:14:37.938546 11269 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6d6c281d6b264ff6a00210b63af3d916"
format_stamp: "Formatted at 2026-08-09 02:14:37 on dist-test-slave-qg60"
server_key: "febcc979865ea0783a2936a1528c549e"
server_key_iv: "42927fd22138558189f216d8a5bebea2"
server_key_version: "encryptionkey@0"
I20260809 02:14:37.939999 11269 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:37.940866 11286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:37.941044 11269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:37.941179 11269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "6d6c281d6b264ff6a00210b63af3d916"
format_stamp: "Formatted at 2026-08-09 02:14:37 on dist-test-slave-qg60"
server_key: "febcc979865ea0783a2936a1528c549e"
server_key_iv: "42927fd22138558189f216d8a5bebea2"
server_key_version: "encryptionkey@0"
I20260809 02:14:37.941262 11269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:37.957154 11269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:37.959169 11269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:37.959334 11269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:37.964408 11269 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45631
I20260809 02:14:37.964483 11357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45631 every 8 connection(s)
I20260809 02:14:37.964879 11269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:37.965734 11360 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:37.967995 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 11269
I20260809 02:14:37.968101 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:37.968273 9377 external_mini_cluster.cc:1428] Setting key d496e353ac748a5210031c8b78a67eb4
I20260809 02:14:37.968283 11360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Bootstrap starting.
I20260809 02:14:37.969044 11360 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:37.969355 11360 log.cc:824] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:37.970224 11360 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: No bootstrap required, opened a new log
I20260809 02:14:37.972605 11360 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } }
I20260809 02:14:37.972836 11360 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:37.972882 11360 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6d6c281d6b264ff6a00210b63af3d916, State: Initialized, Role: FOLLOWER
I20260809 02:14:37.973016 11360 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } }
I20260809 02:14:37.973088 11360 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:37.973127 11360 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:37.973176 11360 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:37.973961 11360 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } }
I20260809 02:14:37.974083 11360 leader_election.cc:302] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6d6c281d6b264ff6a00210b63af3d916; no voters:
I20260809 02:14:37.974279 11360 leader_election.cc:288] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:37.974334 11365 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:37.974521 11365 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [term 1 LEADER]: Becoming Leader. State: Replica: 6d6c281d6b264ff6a00210b63af3d916, State: Running, Role: LEADER
I20260809 02:14:37.974574 11360 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:37.974622 11365 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } }
I20260809 02:14:37.974988 11365 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6d6c281d6b264ff6a00210b63af3d916. Latest consensus state: current_term: 1 leader_uuid: "6d6c281d6b264ff6a00210b63af3d916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } } }
I20260809 02:14:37.975057 11365 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:37.975081 11366 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6d6c281d6b264ff6a00210b63af3d916" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6d6c281d6b264ff6a00210b63af3d916" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45631 } } }
I20260809 02:14:37.975155 11366 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916 [sys.catalog]: This master's current role is: LEADER
Aug 09 02:14:37 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241677, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
W20260809 02:14:37.976887 11384 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:37.976950 11384 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:37.977016 11369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:37.977231 11369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:37.979620 11369 catalog_manager.cc:1347] Generated new cluster ID: 683442fd83a54ba58076508201e9c29e
I20260809 02:14:37.979686 11369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:37.980871 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:37.968821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33216 (local address 127.9.40.126:45631)
0809 02:14:37.968993 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:37.969002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:37.969037 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:14:37.969639 (+ 602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:37.969648 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:37.969712 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:37.969864 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:37.970700 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:37.971578 (+ 878us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:37.972682 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:37.973341 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:37.973432 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:37.976476 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:37.976482 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:37.976483 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:37.976496 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:37.979038 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:37.979539 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:37.979543 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:37.979543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:37.979656 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:37.979910 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:37.979911 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:37.979912 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:37.980160 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:37.980323 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:37.980445 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 02:14:37.980645 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":101,"threads_started":1}
I20260809 02:14:37.984669 11369 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:37.985438 11369 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:37.995355 11369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Generated new TSK 0
I20260809 02:14:37.995575 11369 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:38.033231 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45631
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:38.118077 11390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:38.118311 11390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:38.118338 11390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:38.119480 11390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:38.119524 11390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:38.119544 11390 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:14:38.119590 11390 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:38.119624 11390 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:38.121120 11390 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:38.121368 11390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:38.121603 11390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:38.126792 11395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:38.126803 11390 server_base.cc:1034] running on GCE node
W20260809 02:14:38.126785 11396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:38.126785 11400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:38.127164 11390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:38.127405 11390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:38.128568 11390 hybrid_clock.cc:648] HybridClock initialized: now 1786241678128544 us; error 35 us; skew 500 ppm
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:38.131151 11390 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:38.131637 11390 webserver.cc:460] Webserver started at http://127.9.40.65:33247/ using document root <none> and password file <none>
I20260809 02:14:38.131865 11390 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:38.131938 11390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:38.132091 11390 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:38.133214 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "405ecb4ecca040978e965b53d79f5ea7"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "f6ade9f298b952524f71db2b8d87c12c"
server_key_iv: "45acaee8301f92400e91219ee39ca57f"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.133574 11390 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "405ecb4ecca040978e965b53d79f5ea7"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "f6ade9f298b952524f71db2b8d87c12c"
server_key_iv: "45acaee8301f92400e91219ee39ca57f"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.134986 11390 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:38.135792 11409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.135954 11390 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:38.136057 11390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "405ecb4ecca040978e965b53d79f5ea7"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "f6ade9f298b952524f71db2b8d87c12c"
server_key_iv: "45acaee8301f92400e91219ee39ca57f"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.136134 11390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:38.155424 11390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:38.157831 11390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:38.157997 11390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:38.158615 11390 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:38.158699 11390 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.158743 11390 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:38.158768 11390 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:38.166534 11390 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:43963
I20260809 02:14:38.166955 11390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:38.167779 11535 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:43963 every 8 connection(s)
I20260809 02:14:38.168624 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.159090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51429 (local address 127.9.40.126:45631)
0809 02:14:38.159161 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.159164 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.159411 (+ 247us) server_negotiation.cc:407] Connection header received
0809 02:14:38.159899 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.159901 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.159914 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.160000 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.160650 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.161230 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.162236 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.162451 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.162536 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.165403 (+ 2867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.165405 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.165406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.165414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.167240 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.167697 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.167698 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.167698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.167755 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.168025 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.168026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.168026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.168174 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.168241 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.168473 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.168555 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:14:38.168781 11419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.159058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.65:51429)
0809 02:14:38.159272 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.159313 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.159802 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.160051 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.160054 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.160304 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.160550 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.160555 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.161330 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.161331 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.162139 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.162141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.162550 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.162551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.162676 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.163227 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.163235 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.165306 (+ 2071us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.167375 (+ 2069us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.167381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.167384 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.167632 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.167837 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.167837 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.167837 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.167960 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.168234 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.168239 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.168361 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.168447 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.168570 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":144,"threads_started":1}
I20260809 02:14:38.168916 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 11390
I20260809 02:14:38.169015 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:38.169296 9377 external_mini_cluster.cc:1428] Setting key dc87c3d8b2937878655bf101a7adeb06
I20260809 02:14:38.169541 11536 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45631
I20260809 02:14:38.169632 11536 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:38.169816 11536 heartbeater.cc:511] Master 127.9.40.126:45631 requested a full tablet report, sending...
I20260809 02:14:38.170333 11304 ts_manager.cc:194] Registered new tserver with Master: 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963)
I20260809 02:14:38.171059 11304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:51429
I20260809 02:14:38.177032 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.170671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:46629 (local address 127.9.40.126:45631)
0809 02:14:38.170718 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.170721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.170849 (+ 128us) server_negotiation.cc:407] Connection header received
0809 02:14:38.170938 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.170939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.170948 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.170977 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.171527 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.172153 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.172929 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.173118 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.173197 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.174185 (+ 988us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.174187 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.174187 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.174194 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.175730 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.176155 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.176157 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.176157 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.176219 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.176424 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.176472 (+ 48us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.176473 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.176591 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.176671 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.176858 (+ 187us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.176936 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:38.177047 11542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.170598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.65:46629)
0809 02:14:38.170785 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.170798 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.170875 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.171041 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.171041 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.171354 (+ 313us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.171445 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.171447 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.172254 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.172255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.172838 (+ 583us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.172840 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.173200 (+ 360us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.173201 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.173280 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.173748 (+ 468us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.173752 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.174024 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.175815 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.175816 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.175817 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.176082 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.176292 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.176293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.176293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.176358 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.176676 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.176677 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.176758 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.176822 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.176943 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":85,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:38.200568 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45631
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:38.284456 11545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:38.284693 11545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:38.284737 11545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:38.285992 11545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:38.286051 11545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:38.286085 11545 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:14:38.286114 11545 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:38.286156 11545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:38.287637 11545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:38.287914 11545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:38.288159 11545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:38.293520 11553 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:38.293565 11551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:38.293555 11555 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:38.293787 11545 server_base.cc:1034] running on GCE node
I20260809 02:14:38.293991 11545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:38.294274 11545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:38.295466 11545 hybrid_clock.cc:648] HybridClock initialized: now 1786241678295445 us; error 46 us; skew 500 ppm
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:38.298226 11545 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:38.298691 11545 webserver.cc:460] Webserver started at http://127.9.40.66:46761/ using document root <none> and password file <none>
I20260809 02:14:38.298938 11545 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:38.299010 11545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:38.299154 11545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:38.300220 11545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "0ae6b30758de4875881fad04a8e9ffe2"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "e7440193827fe3876fe904d68b2a74f2"
server_key_iv: "6ef90f106b16809993a9f1d67ffa12aa"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.300586 11545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "0ae6b30758de4875881fad04a8e9ffe2"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "e7440193827fe3876fe904d68b2a74f2"
server_key_iv: "6ef90f106b16809993a9f1d67ffa12aa"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.301971 11545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:38.302748 11564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.302907 11545 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.303056 11545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "0ae6b30758de4875881fad04a8e9ffe2"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "e7440193827fe3876fe904d68b2a74f2"
server_key_iv: "6ef90f106b16809993a9f1d67ffa12aa"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.303133 11545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:38.317880 11545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:38.319757 11545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:38.319936 11545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:38.320482 11545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:38.320549 11545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.320593 11545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:38.320622 11545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.327725 11545 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:42387
I20260809 02:14:38.327811 11693 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:42387 every 8 connection(s)
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:38.328159 11545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:38.331530 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.321026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:60713 (local address 127.9.40.126:45631)
0809 02:14:38.321097 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.321099 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.321298 (+ 199us) server_negotiation.cc:407] Connection header received
0809 02:14:38.321851 (+ 553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.321852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.321866 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.321912 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.322764 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.323634 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.325230 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.325570 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.325643 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.328605 (+ 2962us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.328607 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.328607 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.328616 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.330127 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.330558 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.330559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.330560 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.330630 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.330884 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.330885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.330885 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.331005 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.331078 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.331358 (+ 280us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.331438 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:14:38.331714 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.321003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.66:60713)
0809 02:14:38.321181 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.321222 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.321739 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.321993 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.322002 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.322350 (+ 348us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.322648 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.322655 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.323742 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.323743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.325152 (+ 1409us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.325155 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.325656 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.325657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.325826 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.326497 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.326517 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.328512 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.330237 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.330245 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.330248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.330474 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.330696 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.330697 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.330697 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.330812 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.331069 (+ 257us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.331076 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.331220 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.331322 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.331485 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":107,"threads_started":1}
I20260809 02:14:38.332296 11694 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45631
I20260809 02:14:38.332413 11694 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:38.332649 11694 heartbeater.cc:511] Master 127.9.40.126:45631 requested a full tablet report, sending...
I20260809 02:14:38.333109 11304 ts_manager.cc:194] Registered new tserver with Master: 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387)
I20260809 02:14:38.333565 11304 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:60713
I20260809 02:14:38.336651 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 11545
I20260809 02:14:38.336767 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:38.336961 9377 external_mini_cluster.cc:1428] Setting key cd6e2bb9a855c9ad45c32efca1005ed8
I20260809 02:14:38.339390 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.333439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:34187 (local address 127.9.40.126:45631)
0809 02:14:38.333481 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.333483 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.333601 (+ 118us) server_negotiation.cc:407] Connection header received
0809 02:14:38.333665 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.333666 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.333676 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.333735 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.334120 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.334608 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.335336 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.335509 (+ 173us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.335558 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.336410 (+ 852us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.336412 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.336412 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.336418 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.338162 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.338493 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.338495 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.338495 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.338560 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.338725 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.338725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.338726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.338961 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.339050 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.339255 (+ 205us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.339327 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:14:38.339479 11700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.333385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.66:34187)
0809 02:14:38.333535 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.333547 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.333613 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.333741 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.333742 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.333975 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.334035 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.334037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.334703 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.334705 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.335265 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.335267 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.335573 (+ 306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.335574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.335633 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.336024 (+ 391us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.336028 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.336317 (+ 289us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.338246 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.338247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.338247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.338424 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.338617 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.338618 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.338618 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.338669 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.339084 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.339085 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.339138 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.339207 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.339332 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":92,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:38.366318 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:45631
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:38.450208 11703 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:38.450471 11703 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:38.450511 11703 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:38.451733 11703 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:38.451805 11703 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:38.451840 11703 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260809 02:14:38.451869 11703 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:14:38.451915 11703 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:38.453565 11703 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:45631
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11703
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:38.453861 11703 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:38.454106 11703 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:38.459389 11712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:38.459437 11714 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:38.459419 11711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:38.459817 11703 server_base.cc:1034] running on GCE node
I20260809 02:14:38.460069 11703 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:38.460369 11703 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:38.461565 11703 hybrid_clock.cc:648] HybridClock initialized: now 1786241678461544 us; error 37 us; skew 500 ppm
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:38.464103 11703 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:38.464633 11703 webserver.cc:460] Webserver started at http://127.9.40.67:38977/ using document root <none> and password file <none>
I20260809 02:14:38.464895 11703 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:38.464942 11703 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:38.465044 11703 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:38.466140 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "7cc48337e1b94f86a0d7624be3ca7879"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "8106130f3ca4b34bbe191cf9d10eea4e"
server_key_iv: "af100c43ce43276652a4a06bfc7f5997"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.466449 11703 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "7cc48337e1b94f86a0d7624be3ca7879"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "8106130f3ca4b34bbe191cf9d10eea4e"
server_key_iv: "af100c43ce43276652a4a06bfc7f5997"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.467744 11703 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:38.468673 11722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.468873 11703 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:38.468971 11703 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "7cc48337e1b94f86a0d7624be3ca7879"
format_stamp: "Formatted at 2026-08-09 02:14:38 on dist-test-slave-qg60"
server_key: "8106130f3ca4b34bbe191cf9d10eea4e"
server_key_iv: "af100c43ce43276652a4a06bfc7f5997"
server_key_version: "encryptionkey@0"
I20260809 02:14:38.469028 11703 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:38.492687 11703 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:38.494712 11703 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:38.494830 11703 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:38.495505 11703 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:38.495556 11703 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.495584 11703 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:38.495631 11703 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.502660 11703 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:33617
I20260809 02:14:38.502739 11860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:33617 every 8 connection(s)
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:38.503052 11703 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:38.506016 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.495905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60985 (local address 127.9.40.126:45631)
0809 02:14:38.495954 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.495958 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.496203 (+ 245us) server_negotiation.cc:407] Connection header received
0809 02:14:38.496700 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.496702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.496714 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.496810 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.497630 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.498474 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.500108 (+ 1634us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.500445 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.500523 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.503379 (+ 2856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.503381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.503381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.503390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.504767 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.505152 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.505153 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.505153 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.505204 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.505411 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.505412 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.505412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.505511 (+ 99us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.505561 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.505849 (+ 288us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.505913 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:14:38.506275 11732 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.495892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.67:60985)
0809 02:14:38.496064 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.496105 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.496597 (+ 492us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.496866 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.496869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.497205 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.497511 (+ 306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.497519 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.498584 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.498586 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.499992 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.499996 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.500542 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.500543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.500699 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.501307 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.501318 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.503252 (+ 1934us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.504855 (+ 1603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.504861 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.504863 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.505091 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.505260 (+ 169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.505260 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.505260 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.505360 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.505588 (+ 228us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.505595 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.505721 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.505825 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.506059 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":105,"threads_started":1}
I20260809 02:14:38.506810 11861 heartbeater.cc:347] Connected to a master server at 127.9.40.126:45631
I20260809 02:14:38.506896 11861 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:38.507124 11861 heartbeater.cc:511] Master 127.9.40.126:45631 requested a full tablet report, sending...
I20260809 02:14:38.507519 11305 ts_manager.cc:194] Registered new tserver with Master: 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:38.508001 11305 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60985
I20260809 02:14:38.512593 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 11703
I20260809 02:14:38.512709 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:38.512993 9377 external_mini_cluster.cc:1428] Setting key ab2c3925168e9961943336d3fb24c064
I20260809 02:14:38.513659 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.507994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36123 (local address 127.9.40.126:45631)
0809 02:14:38.508035 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.508037 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.508237 (+ 200us) server_negotiation.cc:407] Connection header received
0809 02:14:38.508287 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.508287 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.508298 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.508352 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.508781 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.509242 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.509928 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.510100 (+ 172us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.510150 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.511057 (+ 907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.511058 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.511058 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.511063 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.512326 (+ 1263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.512816 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.512817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.512817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.512879 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.513096 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.513097 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.513098 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.513246 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.513330 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.513492 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.513563 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:38.513691 11869 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.508009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.9.40.67:36123)
0809 02:14:38.508177 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.508188 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.508245 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.508357 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.508358 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.508610 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.508667 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.508669 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.509357 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.509358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.509854 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.509855 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.510163 (+ 308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.510163 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.510218 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.510696 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.510701 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.510988 (+ 287us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.512452 (+ 1464us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.512453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.512453 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.512683 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.512963 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.512964 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.512964 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.513024 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.513334 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.513335 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.513387 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.513453 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.513566 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":118,"threads_started":1}
I20260809 02:14:38.513839 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241678, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:38.537873 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.530839 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33222 (local address 127.9.40.126:45631)
0809 02:14:38.530903 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.530904 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.530914 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:38.530984 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.530985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.530997 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.531044 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.531584 (+ 540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.532080 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.532958 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.533156 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.533226 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.535495 (+ 2269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.535497 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.535497 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.535504 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.536870 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.537176 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.537177 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.537178 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.537230 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.537386 (+ 156us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.537387 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.537387 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.537489 (+ 102us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.537565 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.537723 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.537787 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:14:38.539722 11305 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:38.541427 11305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:38.548130 11882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.543740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43963 (local address 127.0.0.1:38360)
0809 02:14:38.544131 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.544139 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.544175 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.544828 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.544829 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.544831 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:38.544870 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.544871 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.546047 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.546048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.546972 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.546974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.547884 (+ 910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.547958 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.547975 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.548038 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.548076 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"mutex_wait_us":1,"thread_start_us":118,"threads_started":1}
I20260809 02:14:38.548211 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.544343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.9.40.65:43963)
0809 02:14:38.544653 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.544657 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.544666 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:38.544705 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.544707 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.544723 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.544832 (+ 109us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:38.544987 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.545972 (+ 985us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547057 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.547807 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547884 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.547941 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.548064 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.548101 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":277,"thread_start_us":149,"threads_started":1}
I20260809 02:14:38.548583 11880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.543722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33617 (local address 127.0.0.1:53506)
0809 02:14:38.543877 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.543915 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.543977 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.544411 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.544413 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.544415 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:38.544481 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.544483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.545841 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.545842 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.547025 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.547028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.548345 (+ 1317us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.548346 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.548404 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.548416 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.548467 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.548504 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":9,"thread_start_us":89,"threads_started":1}
I20260809 02:14:38.548629 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.543967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53506 (local address 127.9.40.67:33617)
0809 02:14:38.544257 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.544261 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.544268 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:14:38.544304 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.544305 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.544317 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.544393 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:38.544565 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.545758 (+ 1193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547104 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.547803 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547884 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.547941 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.548498 (+ 557us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.548537 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":77,"threads_started":1}
I20260809 02:14:38.549229 11780 tablet_service.cc:1467] Processing CreateTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de (DEFAULT_TABLE table=test-table [id=02b77d000a7d497ca58fccc9cfad74d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:38.549187 11464 tablet_service.cc:1467] Processing CreateTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de (DEFAULT_TABLE table=test-table [id=02b77d000a7d497ca58fccc9cfad74d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:38.549567 11464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4603aa4e62d4b8f8cd3156c8750b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.549567 11780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4603aa4e62d4b8f8cd3156c8750b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.552021 11881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.543722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.0.0.1:53470)
0809 02:14:38.543874 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.543915 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.543979 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.544497 (+ 518us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.544498 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.544500 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:38.544544 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.544546 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.545696 (+ 1150us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.545697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.547025 (+ 1328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.547028 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.548830 (+ 1802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.548831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.551846 (+ 3015us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.551850 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.551904 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.551949 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":8,"thread_start_us":61,"threads_started":1}
I20260809 02:14:38.552294 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.544116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53470 (local address 127.9.40.66:42387)
0809 02:14:38.544291 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.544296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.544308 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:38.544348 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.544350 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.544367 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.544507 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:38.544917 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.545603 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.547979 (+ 2376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.548745 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.548827 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.548884 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.552144 (+ 3260us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.552195 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":103,"threads_started":1}
I20260809 02:14:38.552940 11621 tablet_service.cc:1467] Processing CreateTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de (DEFAULT_TABLE table=test-table [id=02b77d000a7d497ca58fccc9cfad74d3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:38.553211 11887 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Bootstrap starting.
I20260809 02:14:38.553313 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c4603aa4e62d4b8f8cd3156c8750b7de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.553385 11886 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Bootstrap starting.
I20260809 02:14:38.553951 11887 tablet_bootstrap.cc:654] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.554314 11887 log.cc:824] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:38.554380 11886 tablet_bootstrap.cc:654] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.554690 11886 log.cc:824] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:38.555166 11887 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: No bootstrap required, opened a new log
I20260809 02:14:38.555240 11887 ts_tablet_manager.cc:1397] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:38.555413 11886 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: No bootstrap required, opened a new log
I20260809 02:14:38.555490 11886 ts_tablet_manager.cc:1397] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:38.556910 11891 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Bootstrap starting.
I20260809 02:14:38.557035 11886 raft_consensus.cc:348] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.557210 11886 raft_consensus.cc:374] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.557261 11886 raft_consensus.cc:729] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.557405 11886 consensus_queue.cc:260] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.557677 11886 ts_tablet_manager.cc:1428] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:38.557693 11891 tablet_bootstrap.cc:654] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.557677 11887 raft_consensus.cc:348] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.557814 11861 heartbeater.cc:503] Master 127.9.40.126:45631 was elected leader, sending a full tablet report...
I20260809 02:14:38.557832 11887 raft_consensus.cc:374] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.557869 11887 raft_consensus.cc:729] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.557986 11887 consensus_queue.cc:260] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.558039 11891 log.cc:824] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:38.558267 11887 ts_tablet_manager.cc:1428] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:14:38.558315 11536 heartbeater.cc:503] Master 127.9.40.126:45631 was elected leader, sending a full tablet report...
I20260809 02:14:38.558950 11891 tablet_bootstrap.cc:492] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: No bootstrap required, opened a new log
I20260809 02:14:38.559037 11891 ts_tablet_manager.cc:1397] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:38.560398 11891 raft_consensus.cc:348] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.560537 11891 raft_consensus.cc:374] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.560588 11891 raft_consensus.cc:729] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.560706 11891 consensus_queue.cc:260] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.560974 11891 ts_tablet_manager.cc:1428] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:14:38.561062 11694 heartbeater.cc:503] Master 127.9.40.126:45631 was elected leader, sending a full tablet report...
W20260809 02:14:38.578744 11695 tablet.cc:2369] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:38.611274 11896 raft_consensus.cc:482] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:38.611409 11896 raft_consensus.cc:504] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.611816 11896 leader_election.cc:288] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:38.614986 11883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.612124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46137 (local address 127.9.40.67:33617)
0809 02:14:38.612184 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.612187 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.612223 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:14:38.612249 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.612250 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.612261 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.612305 (+ 44us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:38.612524 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.613175 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614199 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.614602 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614676 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.614686 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.614862 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.614910 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:14:38.614984 11574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.612054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33617 (local address 127.9.40.66:46137)
0809 02:14:38.612114 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.612127 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.612180 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.612335 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.612336 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.612338 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:38.612416 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.612418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.613258 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.613258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.614104 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.614107 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.614695 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.614783 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.614787 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.614844 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.614887 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":20}
I20260809 02:14:38.615096 11885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.612147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35385 (local address 127.9.40.65:43963)
0809 02:14:38.612208 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.612210 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.612529 (+ 319us) server_negotiation.cc:407] Connection header received
0809 02:14:38.612565 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.612566 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.612584 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.612627 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:38.612815 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.613377 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614310 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.614701 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614780 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.614790 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.614968 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.615019 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:14:38.615146 11897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.612060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43963 (local address 127.9.40.66:35385)
0809 02:14:38.612436 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.612445 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.612518 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.612650 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.612651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.612652 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:38.612720 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.612743 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.613460 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.613460 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.614209 (+ 749us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.614212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.614787 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.614788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.614883 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.614887 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.614947 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.614984 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":168,"thread_start_us":145,"threads_started":1}
I20260809 02:14:38.615437 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4603aa4e62d4b8f8cd3156c8750b7de" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7" is_pre_election: true
I20260809 02:14:38.615394 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4603aa4e62d4b8f8cd3156c8750b7de" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879" is_pre_election: true
I20260809 02:14:38.615598 11487 raft_consensus.cc:2454] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 0.
I20260809 02:14:38.615598 11804 raft_consensus.cc:2454] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 0.
I20260809 02:14:38.615837 11565 leader_election.cc:302] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 405ecb4ecca040978e965b53d79f5ea7; no voters:
I20260809 02:14:38.616037 11896 raft_consensus.cc:2781] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:38.616123 11896 raft_consensus.cc:482] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:38.616154 11896 raft_consensus.cc:3037] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.617183 11896 raft_consensus.cc:504] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.617358 11896 leader_election.cc:288] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 election: Requested vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:38.617506 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4603aa4e62d4b8f8cd3156c8750b7de" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7"
I20260809 02:14:38.617539 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c4603aa4e62d4b8f8cd3156c8750b7de" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879"
I20260809 02:14:38.617585 11487 raft_consensus.cc:3037] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.617652 11804 raft_consensus.cc:3037] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.618317 11487 raft_consensus.cc:2454] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 1.
I20260809 02:14:38.618458 11804 raft_consensus.cc:2454] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 1.
I20260809 02:14:38.618491 11565 leader_election.cc:302] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 405ecb4ecca040978e965b53d79f5ea7; no voters:
I20260809 02:14:38.618566 11896 raft_consensus.cc:2781] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:38.618744 11896 raft_consensus.cc:686] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 LEADER]: Becoming Leader. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Running, Role: LEADER
I20260809 02:14:38.618875 11896 consensus_queue.cc:237] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.619841 11305 catalog_manager.cc:5495] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } health_report { overall_health: UNKNOWN } } }
Aug 09 02:14:38 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241677, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
W20260809 02:14:38.667743 11537 tablet.cc:2369] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:38.669047 11363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.661611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33228 (local address 127.9.40.126:45631)
0809 02:14:38.661689 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:38.661691 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:38.661812 (+ 121us) server_negotiation.cc:407] Connection header received
0809 02:14:38.661905 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:38.661906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:38.661916 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:38.661982 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:38.662483 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.662978 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.663692 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.663873 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.663956 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.666280 (+ 2324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:38.666281 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:38.666282 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:38.666290 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:38.667786 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.668252 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.668254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.668254 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.668308 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:38.668493 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:38.668493 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:38.668494 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:38.668602 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:38.668666 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:38.668856 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:14:38.668942 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:14:38.669327 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:38.661553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:45631 (local address 127.0.0.1:33228)
0809 02:14:38.661740 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:38.661749 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:38.661833 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:38.661988 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:38.661989 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:38.662342 (+ 353us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:38.662390 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.662393 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.663065 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.663066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.663606 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:38.663608 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:38.663957 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:38.663958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:38.664015 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:38.664609 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:38.664614 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:38.666187 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:38.667902 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.667913 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.667916 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.668164 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.668370 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:38.668370 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:38.668371 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:38.668436 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:38.668661 (+ 225us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:38.668667 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:38.668717 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:14:38.668840 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 02:14:38.669172 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":68,"thread_start_us":85,"threads_started":1}
I20260809 02:14:38.671459 11305 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:33228:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:14:38.671729 11305 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:38.673502 11464 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb8ac6314ea4ea68fef029ad69187a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9eae30090ea44c3a7f30ff77fe7e716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:38.673529 11621 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb8ac6314ea4ea68fef029ad69187a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9eae30090ea44c3a7f30ff77fe7e716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:38.673709 11464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb8ac6314ea4ea68fef029ad69187a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.673709 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb8ac6314ea4ea68fef029ad69187a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.673647 11780 tablet_service.cc:1467] Processing CreateTablet for tablet 2cb8ac6314ea4ea68fef029ad69187a7 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=d9eae30090ea44c3a7f30ff77fe7e716]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:14:38.673825 11780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2cb8ac6314ea4ea68fef029ad69187a7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:38.675189 11886 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: Bootstrap starting.
I20260809 02:14:38.675319 11887 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: Bootstrap starting.
I20260809 02:14:38.675308 11891 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: Bootstrap starting.
I20260809 02:14:38.675807 11886 tablet_bootstrap.cc:654] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.675803 11887 tablet_bootstrap.cc:654] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.675803 11891 tablet_bootstrap.cc:654] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:38.676538 11886 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: No bootstrap required, opened a new log
I20260809 02:14:38.676589 11886 ts_tablet_manager.cc:1397] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:38.676829 11891 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: No bootstrap required, opened a new log
I20260809 02:14:38.676790 11886 raft_consensus.cc:348] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.676882 11886 raft_consensus.cc:374] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.676885 11891 ts_tablet_manager.cc:1397] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:14:38.676905 11886 raft_consensus.cc:729] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.676949 11887 tablet_bootstrap.cc:492] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: No bootstrap required, opened a new log
I20260809 02:14:38.676965 11886 consensus_queue.cc:260] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.677011 11887 ts_tablet_manager.cc:1397] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:14:38.677057 11886 ts_tablet_manager.cc:1428] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:38.677057 11891 raft_consensus.cc:348] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.677163 11891 raft_consensus.cc:374] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.677193 11891 raft_consensus.cc:729] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.677234 11891 consensus_queue.cc:260] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.677342 11891 ts_tablet_manager.cc:1428] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:38.677326 11887 raft_consensus.cc:348] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.677405 11887 raft_consensus.cc:374] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:38.677435 11887 raft_consensus.cc:729] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Initialized, Role: FOLLOWER
I20260809 02:14:38.677512 11893 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:38.677481 11887 consensus_queue.cc:260] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.677603 11893 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:38.677685 11887 ts_tablet_manager.cc:1428] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:38.677857 11894 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:38.677953 11894 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:38.678239 11896 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:38.678331 11896 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: This TxnStatusTablet replica's current role is: FOLLOWER
W20260809 02:14:38.753693 11862 tablet.cc:2369] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:38.974591 11896 raft_consensus.cc:482] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:38.974689 11896 raft_consensus.cc:504] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.974907 11896 leader_election.cc:288] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:38.975172 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb8ac6314ea4ea68fef029ad69187a7" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7" is_pre_election: true
I20260809 02:14:38.975150 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb8ac6314ea4ea68fef029ad69187a7" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879" is_pre_election: true
I20260809 02:14:38.975270 11487 raft_consensus.cc:2454] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 0.
I20260809 02:14:38.975270 11804 raft_consensus.cc:2454] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 0.
I20260809 02:14:38.975466 11568 leader_election.cc:302] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:38.975587 11896 raft_consensus.cc:2781] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:38.975625 11896 raft_consensus.cc:482] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:38.975643 11896 raft_consensus.cc:3037] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.976490 11896 raft_consensus.cc:504] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.976629 11896 leader_election.cc:288] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 election: Requested vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:38.976809 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb8ac6314ea4ea68fef029ad69187a7" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7"
I20260809 02:14:38.976902 11487 raft_consensus.cc:3037] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.976857 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2cb8ac6314ea4ea68fef029ad69187a7" candidate_uuid: "0ae6b30758de4875881fad04a8e9ffe2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879"
I20260809 02:14:38.976960 11804 raft_consensus.cc:3037] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:38.977531 11487 raft_consensus.cc:2454] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 1.
I20260809 02:14:38.977664 11804 raft_consensus.cc:2454] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0ae6b30758de4875881fad04a8e9ffe2 in term 1.
I20260809 02:14:38.977738 11565 leader_election.cc:302] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 405ecb4ecca040978e965b53d79f5ea7; no voters:
I20260809 02:14:38.977859 11896 raft_consensus.cc:2781] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:38.977922 11896 raft_consensus.cc:686] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 LEADER]: Becoming Leader. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Running, Role: LEADER
I20260809 02:14:38.977978 11896 consensus_queue.cc:237] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:38.978325 11898 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: TxnStatusTablet state changed. Reason: New leader 0ae6b30758de4875881fad04a8e9ffe2. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:38.978418 11898 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:38.978590 11915 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:14:38.978741 11915 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:14:38.978660 11304 catalog_manager.cc:5495] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } health_report { overall_health: UNKNOWN } } }
I20260809 02:14:39.034425 11908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.031211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.0.0.1:53482)
0809 02:14:39.031287 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.031302 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.031354 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.031581 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.031582 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.031584 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:39.031667 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.031670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.032642 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.032643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.033415 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.033418 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.033897 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.033897 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.033947 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.033964 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.034243 (+ 279us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.034251 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.034289 (+ 38us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.034362 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:14:39.034415 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.031304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53482 (local address 127.9.40.66:42387)
0809 02:14:39.031377 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.031379 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.031390 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:39.031437 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.031438 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.031464 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.031546 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.031759 (+ 213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.032539 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.033501 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.033820 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.033906 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.034052 (+ 146us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.034173 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.034256 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.034302 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.034326 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:14:39.036693 11804 raft_consensus.cc:1261] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Refusing update from remote peer 0ae6b30758de4875881fad04a8e9ffe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.036839 11487 raft_consensus.cc:1261] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Refusing update from remote peer 0ae6b30758de4875881fad04a8e9ffe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.037147 11896 consensus_queue.cc:1035] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.037345 11898 consensus_queue.cc:1035] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.038407 11893 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: TxnStatusTablet state changed. Reason: New leader 0ae6b30758de4875881fad04a8e9ffe2. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.038532 11893 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:39.038625 11894 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: TxnStatusTablet state changed. Reason: New leader 0ae6b30758de4875881fad04a8e9ffe2. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.038741 11894 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:39.039764 11896 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.039860 11896 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:39.039911 11918 mvcc.cc:205] Tried to move back new op lower bound from 7316445917331509248 to 7316445917094342656. Current Snapshot: MvccSnapshot[applied={T|T < 7316445917331509248}]
I20260809 02:14:39.040343 11896 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.040444 11896 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 0ae6b30758de4875881fad04a8e9ffe2: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:14:39.040676 11893 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.040872 11893 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 7cc48337e1b94f86a0d7624be3ca7879: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:39.040908 11894 tablet_replica.cc:439] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0ae6b30758de4875881fad04a8e9ffe2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } } }
I20260809 02:14:39.041003 11894 tablet_replica.cc:442] T 2cb8ac6314ea4ea68fef029ad69187a7 P 405ecb4ecca040978e965b53d79f5ea7: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:14:39.041455 11919 mvcc.cc:205] Tried to move back new op lower bound from 7316445917331509248 to 7316445917094342656. Current Snapshot: MvccSnapshot[applied={T|T < 7316445917331509248}]
I20260809 02:14:39.041671 11921 mvcc.cc:205] Tried to move back new op lower bound from 7316445917331509248 to 7316445917094342656. Current Snapshot: MvccSnapshot[applied={T|T < 7316445917331509248}]
I20260809 02:14:39.045375 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.042537 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53486 (local address 127.9.40.66:42387)
0809 02:14:39.042581 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.042583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.042694 (+ 111us) server_negotiation.cc:407] Connection header received
0809 02:14:39.042737 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.042737 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.042747 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.042815 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.042901 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.043663 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.044526 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.044819 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.044893 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.045091 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.045122 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.045190 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.045278 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.045298 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:39.050427 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.047978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:39097 (local address 127.9.40.66:42387)
0809 02:14:39.048027 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.048029 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.048160 (+ 131us) server_negotiation.cc:407] Connection header received
0809 02:14:39.048222 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.048223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.048232 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.048296 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.048437 (+ 141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.048961 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.049724 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.049904 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.049983 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.050188 (+ 205us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.050221 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.050277 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.050336 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.050355 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260809 02:14:39.050491 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.047926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.9.40.66:39097)
0809 02:14:39.048086 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.048097 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.048176 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.048297 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.048298 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.048299 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:39.048354 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.048355 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.049038 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.049039 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.049626 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.049628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.050004 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.050005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.050072 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.050084 (+ 12us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.050278 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.050279 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.050332 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.050379 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"mutex_wait_us":3,"thread_start_us":92,"threads_started":1}
I20260809 02:14:39.053021 11487 raft_consensus.cc:1261] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Refusing update from remote peer 0ae6b30758de4875881fad04a8e9ffe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.053033 11804 raft_consensus.cc:1261] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Refusing update from remote peer 0ae6b30758de4875881fad04a8e9ffe2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.053239 11898 consensus_queue.cc:1035] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.053364 11896 consensus_queue.cc:1035] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.066457 11304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:39.066583 11304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:39.067410 11304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Sending DeleteTablet for 3 replicas of tablet c4603aa4e62d4b8f8cd3156c8750b7de
I20260809 02:14:39.067703 11464 tablet_service.cc:1514] Processing DeleteTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260809 02:14:39.067790 11780 tablet_service.cc:1514] Processing DeleteTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:14:39.067965 11304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:39.068128 11304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:39.068354 11621 tablet_service.cc:1514] Processing DeleteTablet for tablet c4603aa4e62d4b8f8cd3156c8750b7de with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:39 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53470
I20260809 02:14:39.068609 11942 tablet_replica.cc:330] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: stopping tablet replica
I20260809 02:14:39.068786 11942 raft_consensus.cc:2229] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:39.069015 11942 raft_consensus.cc:2258] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:39.069100 11941 tablet_replica.cc:330] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: stopping tablet replica
I20260809 02:14:39.069212 11941 raft_consensus.cc:2229] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:39.069278 11941 raft_consensus.cc:2258] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:39.069414 11940 tablet_replica.cc:330] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: stopping tablet replica
I20260809 02:14:39.069439 11464 tablet_service.cc:1467] Processing CreateTablet for tablet 79b6e009ab85483daaa10b7f8392b882 (DEFAULT_TABLE table=test-table [id=4e020256d92b4213ba011b634fb097b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:39.069525 11940 raft_consensus.cc:2229] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:39.069578 11940 raft_consensus.cc:2258] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:39.069592 11464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b6e009ab85483daaa10b7f8392b882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:39.069772 11941 ts_tablet_manager.cc:1905] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:39.071153 11887 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Bootstrap starting.
I20260809 02:14:39.071295 11940 ts_tablet_manager.cc:1905] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:39.071295 11942 ts_tablet_manager.cc:1905] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:39.071573 11887 tablet_bootstrap.cc:654] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:39.072423 11780 tablet_service.cc:1467] Processing CreateTablet for tablet 79b6e009ab85483daaa10b7f8392b882 (DEFAULT_TABLE table=test-table [id=4e020256d92b4213ba011b634fb097b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:39.072525 11941 ts_tablet_manager.cc:1918] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:39.072603 11941 log.cc:1196] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/c4603aa4e62d4b8f8cd3156c8750b7de
I20260809 02:14:39.072542 11780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b6e009ab85483daaa10b7f8392b882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:39.072896 11941 ts_tablet_manager.cc:1939] T c4603aa4e62d4b8f8cd3156c8750b7de P 7cc48337e1b94f86a0d7624be3ca7879: Deleting consensus metadata
I20260809 02:14:39.073022 11940 ts_tablet_manager.cc:1918] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:39.073081 11940 log.cc:1196] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/c4603aa4e62d4b8f8cd3156c8750b7de
I20260809 02:14:39.073329 11940 ts_tablet_manager.cc:1939] T c4603aa4e62d4b8f8cd3156c8750b7de P 405ecb4ecca040978e965b53d79f5ea7: Deleting consensus metadata
I20260809 02:14:39.073881 11942 ts_tablet_manager.cc:1918] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:39.073926 11886 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Bootstrap starting.
I20260809 02:14:39.073948 11942 log.cc:1196] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/c4603aa4e62d4b8f8cd3156c8750b7de
I20260809 02:14:39.074005 11287 catalog_manager.cc:4841] TS 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963): tablet c4603aa4e62d4b8f8cd3156c8750b7de (table test-table [id=02b77d000a7d497ca58fccc9cfad74d3]) successfully deleted
I20260809 02:14:39.074232 11289 catalog_manager.cc:4841] TS 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617): tablet c4603aa4e62d4b8f8cd3156c8750b7de (table test-table [id=02b77d000a7d497ca58fccc9cfad74d3]) successfully deleted
I20260809 02:14:39.074277 11942 ts_tablet_manager.cc:1939] T c4603aa4e62d4b8f8cd3156c8750b7de P 0ae6b30758de4875881fad04a8e9ffe2: Deleting consensus metadata
I20260809 02:14:39.074349 11886 tablet_bootstrap.cc:654] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:39.074734 11288 catalog_manager.cc:4841] TS 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387): tablet c4603aa4e62d4b8f8cd3156c8750b7de (table test-table [id=02b77d000a7d497ca58fccc9cfad74d3]) successfully deleted
I20260809 02:14:39.074856 11621 tablet_service.cc:1467] Processing CreateTablet for tablet 79b6e009ab85483daaa10b7f8392b882 (DEFAULT_TABLE table=test-table [id=4e020256d92b4213ba011b634fb097b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:39.074960 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 79b6e009ab85483daaa10b7f8392b882. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:39.076261 11891 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Bootstrap starting.
I20260809 02:14:39.076704 11891 tablet_bootstrap.cc:654] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:39.076817 11887 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: No bootstrap required, opened a new log
I20260809 02:14:39.076870 11887 ts_tablet_manager.cc:1397] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260809 02:14:39.077030 11887 raft_consensus.cc:348] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.077100 11887 raft_consensus.cc:374] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:39.077122 11887 raft_consensus.cc:729] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Initialized, Role: FOLLOWER
I20260809 02:14:39.077169 11887 consensus_queue.cc:260] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.077257 11887 ts_tablet_manager.cc:1428] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:39.077427 11891 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: No bootstrap required, opened a new log
I20260809 02:14:39.077479 11891 ts_tablet_manager.cc:1397] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:39.077654 11891 raft_consensus.cc:348] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.077721 11891 raft_consensus.cc:374] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:39.077746 11891 raft_consensus.cc:729] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Initialized, Role: FOLLOWER
I20260809 02:14:39.077792 11891 consensus_queue.cc:260] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.077910 11886 tablet_bootstrap.cc:492] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: No bootstrap required, opened a new log
I20260809 02:14:39.077955 11886 ts_tablet_manager.cc:1397] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260809 02:14:39.078055 11891 ts_tablet_manager.cc:1428] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:39.078127 11886 raft_consensus.cc:348] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.078192 11886 raft_consensus.cc:374] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:39.078212 11886 raft_consensus.cc:729] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Initialized, Role: FOLLOWER
I20260809 02:14:39.078254 11886 consensus_queue.cc:260] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.078329 11886 ts_tablet_manager.cc:1428] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:39.417121 11894 raft_consensus.cc:482] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:39.417255 11894 raft_consensus.cc:504] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.417670 11894 leader_election.cc:288] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:39.420904 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.418075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:47493 (local address 127.9.40.66:42387)
0809 02:14:39.418129 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.418131 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.418144 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:14:39.418223 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.418224 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.418235 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.418322 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:39.418626 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.419169 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420129 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.420524 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420608 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.420621 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.420792 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.420830 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:39.420928 11948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.418064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42757 (local address 127.9.40.67:33617)
0809 02:14:39.418198 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.418200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.418209 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:39.418233 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.418234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.418243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.418322 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:39.418633 (+ 311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.419117 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420118 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.420493 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420577 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.420590 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.420807 (+ 217us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.420853 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":71,"threads_started":1}
I20260809 02:14:39.420946 11946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.417895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33617 (local address 127.9.40.65:42757)
0809 02:14:39.418034 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.418045 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.418105 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.418344 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.418345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.418348 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:39.418446 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.418449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.419224 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.419226 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.420021 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.420023 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.420604 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420606 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.420687 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.420690 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.420794 (+ 104us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.420836 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":78,"thread_start_us":55,"threads_started":1}
I20260809 02:14:39.420967 11947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.417895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.9.40.65:47493)
0809 02:14:39.418032 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.418045 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.418105 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.418344 (+ 239us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.418345 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.418348 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:39.418446 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.418450 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.419294 (+ 844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.419295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.420035 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.420037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.420610 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.420611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.420681 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.420685 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.420779 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.420819 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":19,"thread_start_us":72,"threads_started":1}
I20260809 02:14:39.421233 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b6e009ab85483daaa10b7f8392b882" candidate_uuid: "405ecb4ecca040978e965b53d79f5ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879" is_pre_election: true
I20260809 02:14:39.421236 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b6e009ab85483daaa10b7f8392b882" candidate_uuid: "405ecb4ecca040978e965b53d79f5ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2" is_pre_election: true
I20260809 02:14:39.421336 11804 raft_consensus.cc:2454] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 405ecb4ecca040978e965b53d79f5ea7 in term 0.
I20260809 02:14:39.421360 11642 raft_consensus.cc:2454] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 405ecb4ecca040978e965b53d79f5ea7 in term 0.
I20260809 02:14:39.421521 11412 leader_election.cc:302] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405ecb4ecca040978e965b53d79f5ea7, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:39.421691 11894 raft_consensus.cc:2781] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:39.421736 11894 raft_consensus.cc:482] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:39.421769 11894 raft_consensus.cc:3037] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:39.422503 11894 raft_consensus.cc:504] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.422652 11894 leader_election.cc:288] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [CANDIDATE]: Term 1 election: Requested vote from peers 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387), 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617)
I20260809 02:14:39.422824 11804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b6e009ab85483daaa10b7f8392b882" candidate_uuid: "405ecb4ecca040978e965b53d79f5ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7cc48337e1b94f86a0d7624be3ca7879"
I20260809 02:14:39.422848 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "79b6e009ab85483daaa10b7f8392b882" candidate_uuid: "405ecb4ecca040978e965b53d79f5ea7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2"
I20260809 02:14:39.422906 11804 raft_consensus.cc:3037] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:39.422914 11642 raft_consensus.cc:3037] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:39.423627 11804 raft_consensus.cc:2454] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 405ecb4ecca040978e965b53d79f5ea7 in term 1.
I20260809 02:14:39.423756 11642 raft_consensus.cc:2454] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 405ecb4ecca040978e965b53d79f5ea7 in term 1.
I20260809 02:14:39.423828 11412 leader_election.cc:302] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405ecb4ecca040978e965b53d79f5ea7, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:39.423913 11894 raft_consensus.cc:2781] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:39.424062 11894 raft_consensus.cc:686] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 LEADER]: Becoming Leader. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Running, Role: LEADER
I20260809 02:14:39.424146 11894 consensus_queue.cc:237] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:39.424923 11303 catalog_manager.cc:5495] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "405ecb4ecca040978e965b53d79f5ea7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } health_report { overall_health: UNKNOWN } } }
I20260809 02:14:39.433542 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.431104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38372 (local address 127.9.40.65:43963)
0809 02:14:39.431251 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.431253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.431261 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:39.431299 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.431299 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.431317 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.431398 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.431481 (+ 83us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.431950 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.432799 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.433016 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.433074 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.433259 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.433358 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.433409 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.433462 (+ 53us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.433478 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":100,"threads_started":1}
I20260809 02:14:39.438846 11954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.436132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.9.40.65:58579)
0809 02:14:39.436311 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.436322 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.436374 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.436469 (+ 95us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.436470 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.436471 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:39.436799 (+ 328us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.436801 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.437381 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.437382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.438031 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.438034 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.438372 (+ 338us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.438373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.438437 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.438452 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.438646 (+ 194us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.438647 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.438683 (+ 36us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.438751 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":106,"threads_started":1}
I20260809 02:14:39.438859 11884 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.436213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58579 (local address 127.9.40.66:42387)
0809 02:14:39.436258 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.436260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.436376 (+ 116us) server_negotiation.cc:407] Connection header received
0809 02:14:39.436412 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.436413 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.436421 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.436468 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.436854 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.437298 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.438116 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.438291 (+ 175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.438339 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.438558 (+ 219us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.438591 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.438638 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.438699 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.438737 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:14:39.441182 11804 raft_consensus.cc:1261] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Refusing update from remote peer 405ecb4ecca040978e965b53d79f5ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.441314 11642 raft_consensus.cc:1261] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Refusing update from remote peer 405ecb4ecca040978e965b53d79f5ea7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:39.441444 11949 consensus_queue.cc:1035] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.441558 11894 consensus_queue.cc:1035] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:39.449246 11951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.446858 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:53065 (local address 127.9.40.65:43963)
0809 02:14:39.446896 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:39.446898 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:39.446936 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:14:39.446959 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:39.446960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:39.446972 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:39.447023 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:39.447163 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.447689 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.448409 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.448729 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.448848 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.449002 (+ 154us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.449037 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.449091 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:39.449175 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:14:39.449196 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:14:39.449308 11934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:39.446802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43963 (local address 127.9.40.66:53065)
0809 02:14:39.446841 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:39.446852 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:39.446892 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:39.447034 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:39.447035 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:39.447037 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:39.447083 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.447084 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.447757 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.447758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.448293 (+ 535us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:39.448295 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:39.448846 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:39.448847 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:39.448921 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:39.448929 (+ 8us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:39.449110 (+ 181us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:39.449114 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:39.449168 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:14:39.449225 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":16}
W20260809 02:14:39.449643 11921 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:14:40.973845 11304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:40.974018 11304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33222:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:40.974807 11304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Sending DeleteTablet for 3 replicas of tablet 79b6e009ab85483daaa10b7f8392b882
I20260809 02:14:40.975055 11464 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b6e009ab85483daaa10b7f8392b882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:40 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260809 02:14:40.975075 11621 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b6e009ab85483daaa10b7f8392b882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:40 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53470
I20260809 02:14:40.975119 11780 tablet_service.cc:1514] Processing DeleteTablet for tablet 79b6e009ab85483daaa10b7f8392b882 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:40 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:14:40.975317 11983 tablet_replica.cc:330] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: stopping tablet replica
I20260809 02:14:40.975322 11982 tablet_replica.cc:330] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: stopping tablet replica
I20260809 02:14:40.975375 11982 raft_consensus.cc:2229] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:40.975378 11981 tablet_replica.cc:330] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: stopping tablet replica
I20260809 02:14:40.975427 11981 raft_consensus.cc:2229] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:40.975445 11983 raft_consensus.cc:2229] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:40.975479 11982 raft_consensus.cc:2258] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:40.975533 11983 raft_consensus.cc:2258] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:40.975587 11981 raft_consensus.cc:2258] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:40.975760 11982 ts_tablet_manager.cc:1905] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:40.975848 11983 ts_tablet_manager.cc:1905] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:40.975998 11981 ts_tablet_manager.cc:1905] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:40.977823 11981 ts_tablet_manager.cc:1918] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:14:40.977843 11982 ts_tablet_manager.cc:1918] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:14:40.977898 11982 log.cc:1196] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/79b6e009ab85483daaa10b7f8392b882
I20260809 02:14:40.977984 11981 log.cc:1196] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/79b6e009ab85483daaa10b7f8392b882
I20260809 02:14:40.978106 11983 ts_tablet_manager.cc:1918] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:14:40.978194 11982 ts_tablet_manager.cc:1939] T 79b6e009ab85483daaa10b7f8392b882 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting consensus metadata
I20260809 02:14:40.978221 11983 log.cc:1196] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/79b6e009ab85483daaa10b7f8392b882
I20260809 02:14:40.978344 11981 ts_tablet_manager.cc:1939] T 79b6e009ab85483daaa10b7f8392b882 P 405ecb4ecca040978e965b53d79f5ea7: Deleting consensus metadata
I20260809 02:14:40.978475 11983 ts_tablet_manager.cc:1939] T 79b6e009ab85483daaa10b7f8392b882 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting consensus metadata
I20260809 02:14:40.978674 11287 catalog_manager.cc:4841] TS 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963): tablet 79b6e009ab85483daaa10b7f8392b882 (table test-table [id=4e020256d92b4213ba011b634fb097b1]) successfully deleted
I20260809 02:14:40.978680 11288 catalog_manager.cc:4841] TS 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387): tablet 79b6e009ab85483daaa10b7f8392b882 (table test-table [id=4e020256d92b4213ba011b634fb097b1]) successfully deleted
I20260809 02:14:40.978854 11289 catalog_manager.cc:4841] TS 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617): tablet 79b6e009ab85483daaa10b7f8392b882 (table test-table [id=4e020256d92b4213ba011b634fb097b1]) successfully deleted
Aug 09 02:14:40 dist-test-slave-qg60 krb5kdc[11255](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241680, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:14:40 dist-test-slave-qg60 krb5kdc[11255](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241680, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:41.006649 11993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:40.994002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33234 (local address 127.9.40.126:45631)
0809 02:14:40.994158 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:40.994160 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:40.994172 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:40.994224 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:40.994225 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:40.994235 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:40.994289 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:40.994882 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:40.995413 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:40.996190 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:40.996384 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:40.996436 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.003519 (+ 7083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:41.003520 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:41.003521 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:41.003530 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:41.005332 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:41.005765 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:41.005767 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:41.005767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:41.005838 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:41.006028 (+ 190us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:41.006029 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:41.006029 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:41.006215 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:41.006325 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.006421 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.006513 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260809 02:14:41.008540 11304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:41.008694 11304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:41.010370 11464 tablet_service.cc:1467] Processing CreateTablet for tablet ca769d289143423e9f773a233260663c (DEFAULT_TABLE table=test-table [id=88dc07021e2d437794566a92bff8cbe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.010370 11621 tablet_service.cc:1467] Processing CreateTablet for tablet ca769d289143423e9f773a233260663c (DEFAULT_TABLE table=test-table [id=88dc07021e2d437794566a92bff8cbe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.010428 11780 tablet_service.cc:1467] Processing CreateTablet for tablet ca769d289143423e9f773a233260663c (DEFAULT_TABLE table=test-table [id=88dc07021e2d437794566a92bff8cbe2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.010545 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca769d289143423e9f773a233260663c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.010545 11464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca769d289143423e9f773a233260663c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.010588 11780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ca769d289143423e9f773a233260663c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.011693 11994 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Bootstrap starting.
I20260809 02:14:41.012049 11997 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Bootstrap starting.
I20260809 02:14:41.012250 11994 tablet_bootstrap.cc:654] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.012456 11997 tablet_bootstrap.cc:654] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.012854 11995 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Bootstrap starting.
I20260809 02:14:41.013031 11994 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: No bootstrap required, opened a new log
I20260809 02:14:41.013095 11994 ts_tablet_manager.cc:1397] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.013298 11995 tablet_bootstrap.cc:654] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.013288 11994 raft_consensus.cc:348] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.013388 11994 raft_consensus.cc:374] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.013423 11994 raft_consensus.cc:729] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.013486 11994 consensus_queue.cc:260] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.013684 11994 ts_tablet_manager.cc:1428] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.014029 11995 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: No bootstrap required, opened a new log
I20260809 02:14:41.014096 11995 ts_tablet_manager.cc:1397] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.014161 11997 tablet_bootstrap.cc:492] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: No bootstrap required, opened a new log
I20260809 02:14:41.014216 11997 ts_tablet_manager.cc:1397] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:41.014217 11995 raft_consensus.cc:348] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.014305 11995 raft_consensus.cc:374] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.014335 11995 raft_consensus.cc:729] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.014346 11997 raft_consensus.cc:348] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.014425 11997 raft_consensus.cc:374] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.014451 11997 raft_consensus.cc:729] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.014405 11995 consensus_queue.cc:260] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.014500 11997 consensus_queue.cc:260] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.014593 11997 ts_tablet_manager.cc:1428] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:41.014678 11995 ts_tablet_manager.cc:1428] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.034785 12001 raft_consensus.cc:482] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:41.034878 12001 raft_consensus.cc:504] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.035295 12001 leader_election.cc:288] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387)
I20260809 02:14:41.038925 12003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.035529 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.9.40.67:37433)
0809 02:14:41.035825 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:41.035837 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:41.035908 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:41.036089 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:41.036090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:41.036092 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:41.036180 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.036182 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.036955 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.036956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.037767 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.037769 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.038582 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.038583 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.038666 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.038671 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:14:41.038737 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:14:41.038781 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":107,"threads_started":1}
I20260809 02:14:41.038942 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.035613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37433 (local address 127.9.40.66:42387)
0809 02:14:41.035774 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:41.035777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:41.035905 (+ 128us) server_negotiation.cc:407] Connection header received
0809 02:14:41.035976 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:41.035977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:41.035989 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:41.036079 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:41.036243 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.036858 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.037864 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.038508 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.038568 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.038579 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.038793 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.038861 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":94,"threads_started":1}
I20260809 02:14:41.039216 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca769d289143423e9f773a233260663c" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2" is_pre_election: true
I20260809 02:14:41.039300 12002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.035459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43963 (local address 127.9.40.67:60567)
0809 02:14:41.035680 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:41.035693 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:41.035758 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:41.035984 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:41.035985 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:41.035988 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:41.036062 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.036064 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.036783 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.036785 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.038216 (+ 1431us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.038219 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.039042 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.039043 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.039100 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.039104 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:41.039158 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:14:41.039199 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":146,"threads_started":1}
I20260809 02:14:41.039309 11642 raft_consensus.cc:2454] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 0.
I20260809 02:14:41.039496 11724 leader_election.cc:302] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:41.039546 12006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.035683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60567 (local address 127.9.40.65:43963)
0809 02:14:41.035824 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:41.035826 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:41.035837 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:41.035868 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:41.035869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:41.035883 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:41.035966 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:41.036141 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.036673 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.038305 (+ 1632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.038839 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.038904 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.038915 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.039393 (+ 478us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.039447 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260809 02:14:41.039678 12001 raft_consensus.cc:2781] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:41.039714 12001 raft_consensus.cc:482] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:41.039733 12001 raft_consensus.cc:3037] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.039721 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca769d289143423e9f773a233260663c" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7" is_pre_election: true
I20260809 02:14:41.039789 11487 raft_consensus.cc:2454] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 0.
I20260809 02:14:41.040503 12001 raft_consensus.cc:504] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.040666 12001 leader_election.cc:288] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 election: Requested vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387)
I20260809 02:14:41.040870 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca769d289143423e9f773a233260663c" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2"
I20260809 02:14:41.040903 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ca769d289143423e9f773a233260663c" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7"
I20260809 02:14:41.040969 11642 raft_consensus.cc:3037] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.040989 11487 raft_consensus.cc:3037] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.041613 11642 raft_consensus.cc:2454] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 1.
I20260809 02:14:41.041616 11487 raft_consensus.cc:2454] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 1.
I20260809 02:14:41.041819 11724 leader_election.cc:302] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:41.041960 12001 raft_consensus.cc:2781] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:41.042120 12001 raft_consensus.cc:686] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Running, Role: LEADER
I20260809 02:14:41.042222 12001 consensus_queue.cc:237] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.042949 11303 catalog_manager.cc:5495] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "7cc48337e1b94f86a0d7624be3ca7879" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } health_report { overall_health: HEALTHY } } }
I20260809 02:14:41.053432 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.050814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53514 (local address 127.9.40.67:33617)
0809 02:14:41.050958 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:41.050960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:41.050965 (+ 5us) server_negotiation.cc:407] Connection header received
0809 02:14:41.050996 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:41.050996 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:41.051013 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:41.051079 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:41.051198 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.051672 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.052468 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.052670 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.052776 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.053059 (+ 283us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:41.053168 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:41.053226 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.053290 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.053308 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":95,"threads_started":1}
I20260809 02:14:41.059036 12005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.056404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:52015 (local address 127.9.40.66:42387)
0809 02:14:41.056476 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:41.056478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:41.056489 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:41.056566 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:41.056567 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:41.056582 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:41.056653 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:41.056809 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.057402 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.058251 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.058470 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.058548 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.058752 (+ 204us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:41.058795 (+ 43us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:41.058862 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.058926 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.058949 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:14:41.059072 12015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.056218 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42387 (local address 127.9.40.67:52015)
0809 02:14:41.056390 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:41.056402 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:41.056492 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:41.056656 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:41.056657 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:41.056658 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:41.056717 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.056719 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.057497 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.057499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.058157 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.058159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.058547 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.058548 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.058635 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.058649 (+ 14us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:41.058862 (+ 213us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:41.058864 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:14:41.058918 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:14:41.058966 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260809 02:14:41.061272 11642 raft_consensus.cc:1261] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Refusing update from remote peer 7cc48337e1b94f86a0d7624be3ca7879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:41.061273 11487 raft_consensus.cc:1261] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Refusing update from remote peer 7cc48337e1b94f86a0d7624be3ca7879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:41.061544 12007 consensus_queue.cc:1035] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:41.061671 12001 consensus_queue.cc:1035] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:41.068987 12011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.066471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50213 (local address 127.9.40.67:33617)
0809 02:14:41.066547 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:41.066549 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:41.066717 (+ 168us) server_negotiation.cc:407] Connection header received
0809 02:14:41.066768 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:41.066769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:41.066777 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:41.066819 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:41.066900 (+ 81us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.067346 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.068239 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.068515 (+ 276us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.068614 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.068805 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:41.068835 (+ 30us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:41.068872 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:41.068906 (+ 34us) server_negotiation.cc:299] Negotiation successful
0809 02:14:41.068932 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:14:41.069010 12021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:41.066422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33617 (local address 127.9.40.66:50213)
0809 02:14:41.066670 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:41.066680 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:41.066729 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:41.066820 (+ 91us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:41.066821 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:41.066822 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:14:41.066862 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.066863 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.067422 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.067423 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.068140 (+ 717us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:41.068144 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:41.068619 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:41.068620 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:41.068696 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:41.068700 (+ 4us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:14:41.068873 (+ 173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:14:41.068875 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:14:41.068905 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 02:14:41.068952 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":92,"threads_started":1}
I20260809 02:14:41.075350 11304 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:41.075492 11304 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:41.076189 11304 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Sending DeleteTablet for 3 replicas of tablet ca769d289143423e9f773a233260663c
I20260809 02:14:41.076457 11780 tablet_service.cc:1514] Processing DeleteTablet for tablet ca769d289143423e9f773a233260663c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:41 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:14:41.076457 11621 tablet_service.cc:1514] Processing DeleteTablet for tablet ca769d289143423e9f773a233260663c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:41 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53470
I20260809 02:14:41.076530 11464 tablet_service.cc:1514] Processing DeleteTablet for tablet ca769d289143423e9f773a233260663c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:41 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260809 02:14:41.076556 11983 tablet_replica.cc:330] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: stopping tablet replica
I20260809 02:14:41.076603 11983 raft_consensus.cc:2229] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:41.076615 11982 tablet_replica.cc:330] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: stopping tablet replica
I20260809 02:14:41.076620 11981 tablet_replica.cc:330] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: stopping tablet replica
I20260809 02:14:41.076678 11982 raft_consensus.cc:2229] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:41.076689 11983 raft_consensus.cc:2258] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:41.076699 11981 raft_consensus.cc:2229] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:41.076720 11982 raft_consensus.cc:2258] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:41.076767 11981 raft_consensus.cc:2258] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:41.077148 11981 ts_tablet_manager.cc:1905] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:41.077154 11982 ts_tablet_manager.cc:1905] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:41.077338 11304 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:41.077468 11304 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:41.078210 11983 ts_tablet_manager.cc:1905] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:41.079133 11982 ts_tablet_manager.cc:1918] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:41.079196 11982 log.cc:1196] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/ca769d289143423e9f773a233260663c
I20260809 02:14:41.079424 11982 ts_tablet_manager.cc:1939] T ca769d289143423e9f773a233260663c P 0ae6b30758de4875881fad04a8e9ffe2: Deleting consensus metadata
I20260809 02:14:41.080194 11983 ts_tablet_manager.cc:1918] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:41.080250 11983 log.cc:1196] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/ca769d289143423e9f773a233260663c
I20260809 02:14:41.080439 11780 tablet_service.cc:1467] Processing CreateTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 (DEFAULT_TABLE table=test-table [id=12ee93ddf632466bb6b89898b0d28351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.080472 11983 ts_tablet_manager.cc:1939] T ca769d289143423e9f773a233260663c P 7cc48337e1b94f86a0d7624be3ca7879: Deleting consensus metadata
I20260809 02:14:41.080549 11780 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14d1144e9674b6e9419b1b6d25775e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.080811 11981 ts_tablet_manager.cc:1918] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:14:41.080801 11289 catalog_manager.cc:4841] TS 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617): tablet ca769d289143423e9f773a233260663c (table test-table [id=88dc07021e2d437794566a92bff8cbe2]) successfully deleted
I20260809 02:14:41.080860 11981 log.cc:1196] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/ca769d289143423e9f773a233260663c
I20260809 02:14:41.081071 11981 ts_tablet_manager.cc:1939] T ca769d289143423e9f773a233260663c P 405ecb4ecca040978e965b53d79f5ea7: Deleting consensus metadata
I20260809 02:14:41.081845 11994 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Bootstrap starting.
I20260809 02:14:41.081995 11287 catalog_manager.cc:4841] TS 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963): tablet ca769d289143423e9f773a233260663c (table test-table [id=88dc07021e2d437794566a92bff8cbe2]) successfully deleted
I20260809 02:14:41.082069 11464 tablet_service.cc:1467] Processing CreateTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 (DEFAULT_TABLE table=test-table [id=12ee93ddf632466bb6b89898b0d28351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.082176 11288 catalog_manager.cc:4841] TS 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387): tablet ca769d289143423e9f773a233260663c (table test-table [id=88dc07021e2d437794566a92bff8cbe2]) successfully deleted
I20260809 02:14:41.082199 11464 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14d1144e9674b6e9419b1b6d25775e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.082267 11994 tablet_bootstrap.cc:654] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.082952 11994 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: No bootstrap required, opened a new log
I20260809 02:14:41.082994 11994 ts_tablet_manager.cc:1397] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.083168 11994 raft_consensus.cc:348] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.083231 11994 raft_consensus.cc:374] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.083256 11994 raft_consensus.cc:729] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.083302 11994 consensus_queue.cc:260] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.083428 11994 ts_tablet_manager.cc:1428] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:41.083487 11997 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: Bootstrap starting.
I20260809 02:14:41.083510 11621 tablet_service.cc:1467] Processing CreateTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 (DEFAULT_TABLE table=test-table [id=12ee93ddf632466bb6b89898b0d28351]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:41.083619 11621 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d14d1144e9674b6e9419b1b6d25775e5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:41.083817 11997 tablet_bootstrap.cc:654] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.084415 11997 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: No bootstrap required, opened a new log
I20260809 02:14:41.084458 11997 ts_tablet_manager.cc:1397] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.084564 11997 raft_consensus.cc:348] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.084645 11997 raft_consensus.cc:374] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.084647 11995 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Bootstrap starting.
I20260809 02:14:41.084671 11997 raft_consensus.cc:729] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 405ecb4ecca040978e965b53d79f5ea7, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.084713 11997 consensus_queue.cc:260] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.084842 11997 ts_tablet_manager.cc:1428] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:41.085055 11995 tablet_bootstrap.cc:654] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:41.085657 11995 tablet_bootstrap.cc:492] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: No bootstrap required, opened a new log
I20260809 02:14:41.085717 11995 ts_tablet_manager.cc:1397] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:41.085832 11995 raft_consensus.cc:348] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.085903 11995 raft_consensus.cc:374] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:41.085918 11995 raft_consensus.cc:729] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ae6b30758de4875881fad04a8e9ffe2, State: Initialized, Role: FOLLOWER
I20260809 02:14:41.085989 11995 consensus_queue.cc:260] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.086097 11995 ts_tablet_manager.cc:1428] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:41.272014 12018 raft_consensus.cc:482] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:41.272145 12018 raft_consensus.cc:504] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.272403 12018 leader_election.cc:288] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387)
I20260809 02:14:41.272579 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14d1144e9674b6e9419b1b6d25775e5" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7" is_pre_election: true
I20260809 02:14:41.272696 11487 raft_consensus.cc:2454] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 0.
I20260809 02:14:41.272650 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14d1144e9674b6e9419b1b6d25775e5" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2" is_pre_election: true
I20260809 02:14:41.272768 11642 raft_consensus.cc:2454] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 0.
I20260809 02:14:41.272907 11723 leader_election.cc:302] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 405ecb4ecca040978e965b53d79f5ea7, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:41.273043 12018 raft_consensus.cc:2781] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:41.273097 12018 raft_consensus.cc:482] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:41.273118 12018 raft_consensus.cc:3037] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.273867 12018 raft_consensus.cc:504] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.273996 12018 leader_election.cc:288] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 election: Requested vote from peers 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963), 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387)
I20260809 02:14:41.274140 11642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14d1144e9674b6e9419b1b6d25775e5" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0ae6b30758de4875881fad04a8e9ffe2"
I20260809 02:14:41.274137 11487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d14d1144e9674b6e9419b1b6d25775e5" candidate_uuid: "7cc48337e1b94f86a0d7624be3ca7879" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "405ecb4ecca040978e965b53d79f5ea7"
I20260809 02:14:41.274212 11642 raft_consensus.cc:3037] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.274230 11487 raft_consensus.cc:3037] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:41.274905 11642 raft_consensus.cc:2454] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 1.
I20260809 02:14:41.274964 11487 raft_consensus.cc:2454] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7cc48337e1b94f86a0d7624be3ca7879 in term 1.
I20260809 02:14:41.275069 11724 leader_election.cc:302] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0ae6b30758de4875881fad04a8e9ffe2, 7cc48337e1b94f86a0d7624be3ca7879; no voters:
I20260809 02:14:41.275169 12018 raft_consensus.cc:2781] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:41.275228 12018 raft_consensus.cc:686] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 LEADER]: Becoming Leader. State: Replica: 7cc48337e1b94f86a0d7624be3ca7879, State: Running, Role: LEADER
I20260809 02:14:41.275281 12018 consensus_queue.cc:237] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } }
I20260809 02:14:41.275872 11304 catalog_manager.cc:5495] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 reported cstate change: term changed from 0 to 1, leader changed from <none> to 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "7cc48337e1b94f86a0d7624be3ca7879" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7cc48337e1b94f86a0d7624be3ca7879" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33617 } health_report { overall_health: HEALTHY } } }
I20260809 02:14:41.317250 11642 raft_consensus.cc:1261] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Refusing update from remote peer 7cc48337e1b94f86a0d7624be3ca7879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:41.317274 11487 raft_consensus.cc:1261] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Refusing update from remote peer 7cc48337e1b94f86a0d7624be3ca7879: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:41.317535 12018 consensus_queue.cc:1035] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0ae6b30758de4875881fad04a8e9ffe2" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42387 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:41.317731 12019 consensus_queue.cc:1035] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [LEADER]: Connected to new peer: Peer: permanent_uuid: "405ecb4ecca040978e965b53d79f5ea7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43963 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:14:41.322250 11964 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260809 02:14:42.840631 11303 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:42.840848 11303 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33234:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:42.841775 11303 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 6d6c281d6b264ff6a00210b63af3d916: Sending DeleteTablet for 3 replicas of tablet d14d1144e9674b6e9419b1b6d25775e5
I20260809 02:14:42.842027 11464 tablet_service.cc:1514] Processing DeleteTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260809 02:14:42.842284 12054 tablet_replica.cc:330] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7: stopping tablet replica
I20260809 02:14:42.842312 11621 tablet_service.cc:1514] Processing DeleteTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53470
I20260809 02:14:42.842372 12054 raft_consensus.cc:2229] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:42.842422 12054 raft_consensus.cc:2258] T d14d1144e9674b6e9419b1b6d25775e5 P 405ecb4ecca040978e965b53d79f5ea7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:42.842446 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 11390
I20260809 02:14:42.842949 11780 tablet_service.cc:1514] Processing DeleteTablet for tablet d14d1144e9674b6e9419b1b6d25775e5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:42 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53506
I20260809 02:14:42.845191 12055 tablet_replica.cc:330] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: stopping tablet replica
I20260809 02:14:42.845270 12055 raft_consensus.cc:2229] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:42.845347 12055 raft_consensus.cc:2258] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:42.845495 12056 tablet_replica.cc:330] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: stopping tablet replica
I20260809 02:14:42.845573 12056 raft_consensus.cc:2229] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:42.845657 12056 raft_consensus.cc:2258] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:42.845742 12055 ts_tablet_manager.cc:1905] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:42.845981 12056 ts_tablet_manager.cc:1905] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:42.847699 12056 ts_tablet_manager.cc:1918] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:14:42.847754 12055 ts_tablet_manager.cc:1918] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260809 02:14:42.847774 12056 log.cc:1196] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/d14d1144e9674b6e9419b1b6d25775e5
I20260809 02:14:42.847810 12055 log.cc:1196] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/d14d1144e9674b6e9419b1b6d25775e5
I20260809 02:14:42.848058 12055 ts_tablet_manager.cc:1939] T d14d1144e9674b6e9419b1b6d25775e5 P 0ae6b30758de4875881fad04a8e9ffe2: Deleting consensus metadata
I20260809 02:14:42.848058 12056 ts_tablet_manager.cc:1939] T d14d1144e9674b6e9419b1b6d25775e5 P 7cc48337e1b94f86a0d7624be3ca7879: Deleting consensus metadata
I20260809 02:14:42.848701 11288 catalog_manager.cc:4841] TS 0ae6b30758de4875881fad04a8e9ffe2 (127.9.40.66:42387): tablet d14d1144e9674b6e9419b1b6d25775e5 (table test-table [id=12ee93ddf632466bb6b89898b0d28351]) successfully deleted
I20260809 02:14:42.848744 11289 catalog_manager.cc:4841] TS 7cc48337e1b94f86a0d7624be3ca7879 (127.9.40.67:33617): tablet d14d1144e9674b6e9419b1b6d25775e5 (table test-table [id=12ee93ddf632466bb6b89898b0d28351]) successfully deleted
W20260809 02:14:42.852990 11287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:43963 (error 108)
I20260809 02:14:42.853675 12058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:42.853439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43963 (local address 127.0.0.1:38386)
0809 02:14:42.853573 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:42.853631 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43963: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102,"thread_start_us":64,"threads_started":1}
I20260809 02:14:42.853688 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 11545
W20260809 02:14:42.853789 11287 catalog_manager.cc:4584] TS 405ecb4ecca040978e965b53d79f5ea7 (127.9.40.65:43963): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d14d1144e9674b6e9419b1b6d25775e5: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43963: connect: Connection refused (error 111)
I20260809 02:14:42.862807 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 11703
I20260809 02:14:42.870730 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 11269
2026-08-09T02:14:42Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6266 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:42 dist-test-slave-qg60 krb5kdc[12061](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:42 dist-test-slave-qg60 krb5kdc[12061](info): set up 2 sockets
Aug 09 02:14:42 dist-test-slave-qg60 krb5kdc[12061](info): commencing operation
krb5kdc: starting...
W20260809 02:14:43.961872 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:44.092901 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40683
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40683
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:44.175773 12076 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:44.176021 12076 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:44.176059 12076 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:44.177392 12076 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:44.177462 12076 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:44.177486 12076 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:44.177515 12076 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:44.177543 12076 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:44.179044 12076 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40683
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40683
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12076
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:44.179332 12076 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:44.179556 12076 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:44.184907 12085 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.184904 12081 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:44.184924 12076 server_base.cc:1034] running on GCE node
W20260809 02:14:44.184942 12083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:44.185426 12076 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:44.185734 12076 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:44.186880 12076 hybrid_clock.cc:648] HybridClock initialized: now 1786241684186861 us; error 32 us; skew 500 ppm
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:44.189699 12076 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:44.190236 12076 webserver.cc:460] Webserver started at http://127.9.40.126:44019/ using document root <none> and password file <none>
I20260809 02:14:44.190466 12076 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:44.190513 12076 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:44.190685 12076 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:44.191803 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "4c424593fcb74afd9268e3340c14b050"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "c3396d19fa202b0c386728c87cd344cf"
server_key_iv: "fd6b1284b4b7992b55ffe74ea65964df"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.192183 12076 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "4c424593fcb74afd9268e3340c14b050"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "c3396d19fa202b0c386728c87cd344cf"
server_key_iv: "fd6b1284b4b7992b55ffe74ea65964df"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.193679 12076 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:44.194480 12095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.194653 12076 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:44.194757 12076 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "4c424593fcb74afd9268e3340c14b050"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "c3396d19fa202b0c386728c87cd344cf"
server_key_iv: "fd6b1284b4b7992b55ffe74ea65964df"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.194840 12076 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:44.216180 12076 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:44.218675 12076 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:44.218847 12076 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:44.223469 12076 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:40683
I20260809 02:14:44.223531 12153 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:40683 every 8 connection(s)
I20260809 02:14:44.223905 12076 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:44.224911 12154 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:44.227393 12154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: Bootstrap starting.
I20260809 02:14:44.228027 12154 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:44.228351 12154 log.cc:824] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:44.229053 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12076
I20260809 02:14:44.229156 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:44.229332 12154 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: No bootstrap required, opened a new log
I20260809 02:14:44.229346 9377 external_mini_cluster.cc:1428] Setting key e9134733d00a0126124d02e256f96ee5
I20260809 02:14:44.231359 12154 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } }
I20260809 02:14:44.231549 12154 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:44.231618 12154 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c424593fcb74afd9268e3340c14b050, State: Initialized, Role: FOLLOWER
I20260809 02:14:44.231779 12154 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } }
I20260809 02:14:44.231848 12154 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:44.231871 12154 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:44.231925 12154 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:44.232901 12154 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } }
I20260809 02:14:44.233026 12154 leader_election.cc:302] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4c424593fcb74afd9268e3340c14b050; no voters:
I20260809 02:14:44.233237 12154 leader_election.cc:288] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:44.233273 12159 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:44.233393 12159 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [term 1 LEADER]: Becoming Leader. State: Replica: 4c424593fcb74afd9268e3340c14b050, State: Running, Role: LEADER
I20260809 02:14:44.233554 12154 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:44.233543 12159 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } }
I20260809 02:14:44.234131 12161 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4c424593fcb74afd9268e3340c14b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } } }
I20260809 02:14:44.234416 12161 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:44.234333 12160 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4c424593fcb74afd9268e3340c14b050. Latest consensus state: current_term: 1 leader_uuid: "4c424593fcb74afd9268e3340c14b050" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4c424593fcb74afd9268e3340c14b050" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40683 } } }
I20260809 02:14:44.234861 12160 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050 [sys.catalog]: This master's current role is: LEADER
W20260809 02:14:44.235539 12176 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:44.235599 12176 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:44.235987 12169 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:44.236208 12169 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:44.237967 12169 catalog_manager.cc:1347] Generated new cluster ID: 6de0ebb5702e4fbab0d638a5a08e8db9
I20260809 02:14:44.238023 12169 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:44.241647 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.229899 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37504 (local address 127.9.40.126:40683)
0809 02:14:44.230095 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:44.230104 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:44.230130 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:14:44.231341 (+ 1211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:44.231353 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:44.231427 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:44.231642 (+ 215us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:44.232494 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.233713 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.234759 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.235403 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.235527 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.238262 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:44.238266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:44.238266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:44.238279 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:44.240021 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.240406 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.240409 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.240409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.240518 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.240750 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.240751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.240752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.241007 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:44.241200 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:44.241268 (+ 68us) server_negotiation.cc:299] Negotiation successful
0809 02:14:44.241411 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":105,"threads_started":1}
I20260809 02:14:44.252336 12169 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:44.252964 12169 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:44.257647 12169 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4c424593fcb74afd9268e3340c14b050: Generated new TSK 0
I20260809 02:14:44.257848 12169 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:44.296046 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:44.396418 12181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:44.396637 12181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:44.396661 12181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:44.397966 12181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:44.398011 12181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:44.398039 12181 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:44.399492 12181 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:44.399757 12181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:44.399988 12181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:44.405014 12192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.405125 12187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.405020 12189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:44.405296 12181 server_base.cc:1034] running on GCE node
I20260809 02:14:44.405535 12181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:44.405755 12181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:44.406914 12181 hybrid_clock.cc:648] HybridClock initialized: now 1786241684406900 us; error 29 us; skew 500 ppm
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:44.409588 12181 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:44.410066 12181 webserver.cc:460] Webserver started at http://127.9.40.65:41551/ using document root <none> and password file <none>
I20260809 02:14:44.410300 12181 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:44.410344 12181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:44.410470 12181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:44.411554 12181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "b139e1f55b9041cf8806997de150b9ac"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "090fed0449c10da3ffa93d1ce75a112f"
server_key_iv: "bd76360576a3687d73e1499fed7dc71e"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.411948 12181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "b139e1f55b9041cf8806997de150b9ac"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "090fed0449c10da3ffa93d1ce75a112f"
server_key_iv: "bd76360576a3687d73e1499fed7dc71e"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.413465 12181 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 02:14:44.414230 12199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.414402 12181 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:44.414499 12181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "b139e1f55b9041cf8806997de150b9ac"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "090fed0449c10da3ffa93d1ce75a112f"
server_key_iv: "bd76360576a3687d73e1499fed7dc71e"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.414575 12181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:44.446908 12181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:44.448961 12181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:44.449127 12181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:44.449388 12181 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:44.449702 12181 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:44.449736 12181 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.449759 12181 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:44.449802 12181 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.455626 12181 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:40321
I20260809 02:14:44.455706 12334 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:40321 every 8 connection(s)
I20260809 02:14:44.456023 12181 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:44.462460 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12181
I20260809 02:14:44.462596 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:44.462809 9377 external_mini_cluster.cc:1428] Setting key 2325c72e63eb2789d5831736cd703b05
I20260809 02:14:44.465564 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.456783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36245 (local address 127.9.40.126:40683)
0809 02:14:44.456836 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:44.456838 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:44.457050 (+ 212us) server_negotiation.cc:407] Connection header received
0809 02:14:44.457607 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:44.457609 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:44.457625 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:44.457682 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:44.458675 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.459179 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.460183 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.460391 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.460465 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.462640 (+ 2175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:44.462642 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:44.462642 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:44.462653 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:44.464140 (+ 1487us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.464558 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.464559 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.464559 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.464621 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.464911 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.464912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.464912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.465031 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:44.465106 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:44.465382 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 02:14:44.465475 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:14:44.465751 12337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.456713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40683 (local address 127.9.40.65:36245)
0809 02:14:44.456922 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:44.456964 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:44.457500 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:44.457770 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:44.457773 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:44.458354 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:44.458550 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.458554 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.459272 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.459273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.460069 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.460071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.460475 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.460477 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.460603 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.461079 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:44.461086 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:44.462525 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:44.464248 (+ 1723us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.464256 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.464258 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.464480 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.464694 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.464695 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.464695 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.464841 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.465097 (+ 256us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:44.465104 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:44.465248 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 02:14:44.465347 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:14:44.465506 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":118,"threads_started":1}
I20260809 02:14:44.466236 12335 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40683
I20260809 02:14:44.466347 12335 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:44.466529 12335 heartbeater.cc:511] Master 127.9.40.126:40683 requested a full tablet report, sending...
I20260809 02:14:44.467073 12112 ts_manager.cc:194] Registered new tserver with Master: b139e1f55b9041cf8806997de150b9ac (127.9.40.65:40321)
I20260809 02:14:44.467993 12112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36245
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:44.491837 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:44.575300 12340 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:44.575493 12340 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:44.575517 12340 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:44.576673 12340 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:44.576771 12340 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:44.576822 12340 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:44.578421 12340 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12340
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:44.578689 12340 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:44.578917 12340 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:44.584028 12349 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.583901 12347 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.583913 12346 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:44.583968 12340 server_base.cc:1034] running on GCE node
I20260809 02:14:44.584514 12340 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:44.584800 12340 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:44.586027 12340 hybrid_clock.cc:648] HybridClock initialized: now 1786241684586000 us; error 38 us; skew 500 ppm
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:44.588598 12340 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:44.589161 12340 webserver.cc:460] Webserver started at http://127.9.40.66:37295/ using document root <none> and password file <none>
I20260809 02:14:44.589389 12340 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:44.589454 12340 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:44.589622 12340 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:44.591114 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "e0b0c0550d364e6282184e68984bd214"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "711d3d07da52211985161ef44091d6de"
server_key_iv: "20a17671eda516a6497d8f916c35b696"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.591471 12340 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "e0b0c0550d364e6282184e68984bd214"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "711d3d07da52211985161ef44091d6de"
server_key_iv: "20a17671eda516a6497d8f916c35b696"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.592823 12340 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:44.593611 12357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.593798 12340 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:44.593894 12340 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "e0b0c0550d364e6282184e68984bd214"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "711d3d07da52211985161ef44091d6de"
server_key_iv: "20a17671eda516a6497d8f916c35b696"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.593976 12340 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:44.602517 12340 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:44.604249 12340 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:44.604399 12340 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:44.604619 12340 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:44.604972 12340 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:44.605026 12340 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.605068 12340 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:44.605094 12340 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.611282 12340 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:46195
I20260809 02:14:44.611335 12485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:46195 every 8 connection(s)
I20260809 02:14:44.611641 12340 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:44.617408 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12340
I20260809 02:14:44.617519 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:44.617767 9377 external_mini_cluster.cc:1428] Setting key 5b37172df0780b33af3c34de6abbfcf4
I20260809 02:14:44.621353 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.612355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:53949 (local address 127.9.40.126:40683)
0809 02:14:44.612403 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:44.612405 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:44.612593 (+ 188us) server_negotiation.cc:407] Connection header received
0809 02:14:44.613131 (+ 538us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:44.613132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:44.613142 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:44.613196 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:44.614105 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.614614 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.615580 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.615761 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.615814 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.618163 (+ 2349us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:44.618165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:44.618165 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:44.618173 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:44.619712 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.620217 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.620218 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.620218 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.620294 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.620610 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.620611 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.620611 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.620812 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:44.620889 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:44.621138 (+ 249us) server_negotiation.cc:299] Negotiation successful
0809 02:14:44.621243 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:44.621390 12488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.612321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40683 (local address 127.9.40.66:53949)
0809 02:14:44.612473 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:44.612507 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:44.613043 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:44.613247 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:44.613250 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:44.613835 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:44.614025 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.614030 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.614695 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.614695 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.615489 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.615491 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.615833 (+ 342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.615834 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.615929 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.616451 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:44.616459 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:44.618039 (+ 1580us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:44.619794 (+ 1755us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.619802 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.619804 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.620133 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.620381 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.620382 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.620383 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.620541 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.620880 (+ 339us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:44.620886 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:44.621009 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 02:14:44.621096 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:14:44.621226 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":90,"threads_started":1}
I20260809 02:14:44.621912 12486 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40683
I20260809 02:14:44.622020 12486 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:44.622191 12486 heartbeater.cc:511] Master 127.9.40.126:40683 requested a full tablet report, sending...
I20260809 02:14:44.622581 12112 ts_manager.cc:194] Registered new tserver with Master: e0b0c0550d364e6282184e68984bd214 (127.9.40.66:46195)
I20260809 02:14:44.622964 12112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:53949
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:44.645064 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:44.729638 12491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:44.729943 12491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:44.729990 12491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:44.732043 12491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:44.732113 12491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:44.732157 12491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:44.734815 12491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40683
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:44.735119 12491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:44.735432 12491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:44.742266 12496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.742323 12500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:44.742281 12498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:44.742720 12491 server_base.cc:1034] running on GCE node
I20260809 02:14:44.742956 12491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:44.743731 12491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:44.744902 12491 hybrid_clock.cc:648] HybridClock initialized: now 1786241684744884 us; error 31 us; skew 500 ppm
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:44.748440 12491 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:44.749146 12491 webserver.cc:460] Webserver started at http://127.9.40.67:35527/ using document root <none> and password file <none>
I20260809 02:14:44.749445 12491 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:44.749524 12491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:44.749668 12491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:44.751336 12491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "8833a71af13048608a0297a12252bc45"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "7943a73e34a012f9c34f379be393ee9e"
server_key_iv: "a36a33b28a2704f9c47e8f0c7a5e9bc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.751808 12491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "8833a71af13048608a0297a12252bc45"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "7943a73e34a012f9c34f379be393ee9e"
server_key_iv: "a36a33b28a2704f9c47e8f0c7a5e9bc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.753984 12491 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260809 02:14:44.755043 12507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.755259 12491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:44.755355 12491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "8833a71af13048608a0297a12252bc45"
format_stamp: "Formatted at 2026-08-09 02:14:44 on dist-test-slave-qg60"
server_key: "7943a73e34a012f9c34f379be393ee9e"
server_key_iv: "a36a33b28a2704f9c47e8f0c7a5e9bc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:44.755445 12491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:44.769024 12491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:44.771478 12491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:44.771672 12491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:44.771992 12491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:44.772369 12491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:44.772415 12491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.772446 12491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:44.772470 12491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:44.779803 12491 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:37351
I20260809 02:14:44.779847 12633 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:37351 every 8 connection(s)
I20260809 02:14:44.780193 12491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:44.780685 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12491
I20260809 02:14:44.780846 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:44.781064 9377 external_mini_cluster.cc:1428] Setting key 53698d141e8a38d3e9651db1c9b9c4b4
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12061](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241684, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:44.790011 12158 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.781015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59995 (local address 127.9.40.126:40683)
0809 02:14:44.781104 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:44.781106 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:44.781342 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:14:44.782022 (+ 680us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:44.782023 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:44.782036 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:44.782077 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:44.783151 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.783710 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.784910 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.785139 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.785202 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.787422 (+ 2220us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:44.787424 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:44.787424 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:44.787432 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:44.788702 (+ 1270us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.789189 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.789190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.789190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.789242 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:44.789461 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:44.789462 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:44.789462 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:44.789588 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:44.789643 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:44.789883 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:14:44.789945 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260809 02:14:44.790283 12637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:44.781006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40683 (local address 127.9.40.67:59995)
0809 02:14:44.781212 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:44.781267 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:44.781896 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:44.782147 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:44.782150 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:44.782725 (+ 575us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:44.782923 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.782927 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.783820 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.783821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.784833 (+ 1012us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:44.784836 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:44.785217 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:44.785218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:44.785315 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:44.785794 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:44.785802 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:44.787309 (+ 1507us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:44.788838 (+ 1529us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.788844 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.788846 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.789115 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.789298 (+ 183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:44.789299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:44.789299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:44.789404 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:44.789646 (+ 242us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:44.789650 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:44.789769 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 02:14:44.789858 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:14:44.789985 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260809 02:14:44.790864 12635 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40683
I20260809 02:14:44.791003 12635 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:44.791185 12635 heartbeater.cc:511] Master 127.9.40.126:40683 requested a full tablet report, sending...
I20260809 02:14:44.791595 12112 ts_manager.cc:194] Registered new tserver with Master: 8833a71af13048608a0297a12252bc45 (127.9.40.67:37351)
I20260809 02:14:44.792002 12112 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:59995
I20260809 02:14:44.792932 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:14:44.806996 12158 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:44.806145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37510 (local address 127.9.40.126:40683)
0809 02:14:44.806211 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:44.806213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:44.806241 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:14:44.806313 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:44.806314 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:44.806324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:44.806388 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:44.806938 (+ 550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37510: BlockingRecv error: recv got EOF from 127.0.0.1:37510 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
I20260809 02:14:44.807847 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12181
I20260809 02:14:44.817952 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12340
I20260809 02:14:44.825321 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12491
I20260809 02:14:44.832342 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12076
2026-08-09T02:14:44Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1958 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12648](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12648](info): set up 2 sockets
Aug 09 02:14:44 dist-test-slave-qg60 krb5kdc[12648](info): commencing operation
krb5kdc: starting...
W20260809 02:14:45.953613 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.099s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:46.080905 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45217
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45217
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:46.163748 12663 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:46.164008 12663 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:46.164050 12663 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:46.165275 12663 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:46.165347 12663 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:46.165372 12663 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:46.165403 12663 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:46.165432 12663 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:46.166989 12663 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45217
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12663
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:46.167292 12663 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:46.167529 12663 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:46.172349 12672 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:46.172410 12670 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:46.172361 12669 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:46.172861 12663 server_base.cc:1034] running on GCE node
I20260809 02:14:46.173172 12663 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:46.173475 12663 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:46.174633 12663 hybrid_clock.cc:648] HybridClock initialized: now 1786241686174600 us; error 51 us; skew 500 ppm
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12648](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241686, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:46.183252 12663 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:46.183800 12663 webserver.cc:460] Webserver started at http://127.9.40.126:39913/ using document root <none> and password file <none>
I20260809 02:14:46.184041 12663 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:46.184110 12663 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:46.184270 12663 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:46.185482 12663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "bd60f1bf463d478ea57308d77ab8b8c5"
format_stamp: "Formatted at 2026-08-09 02:14:46 on dist-test-slave-qg60"
server_key: "c33b01b49b12402c0057aacb0c3ad6b4"
server_key_iv: "96f38ba83bb7e6dc394081666f6047c5"
server_key_version: "encryptionkey@0"
I20260809 02:14:46.185856 12663 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bd60f1bf463d478ea57308d77ab8b8c5"
format_stamp: "Formatted at 2026-08-09 02:14:46 on dist-test-slave-qg60"
server_key: "c33b01b49b12402c0057aacb0c3ad6b4"
server_key_iv: "96f38ba83bb7e6dc394081666f6047c5"
server_key_version: "encryptionkey@0"
I20260809 02:14:46.187263 12663 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:46.188006 12680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:46.188176 12663 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:46.188279 12663 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "bd60f1bf463d478ea57308d77ab8b8c5"
format_stamp: "Formatted at 2026-08-09 02:14:46 on dist-test-slave-qg60"
server_key: "c33b01b49b12402c0057aacb0c3ad6b4"
server_key_iv: "96f38ba83bb7e6dc394081666f6047c5"
server_key_version: "encryptionkey@0"
I20260809 02:14:46.188350 12663 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:46.211514 12663 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:46.213593 12663 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:46.213768 12663 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:46.218109 12663 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45217
I20260809 02:14:46.218155 12739 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45217 every 8 connection(s)
I20260809 02:14:46.218533 12663 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:46.219475 12741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:46.222047 12741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5: Bootstrap starting.
I20260809 02:14:46.222651 12741 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:46.222934 12741 log.cc:824] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:46.223675 12741 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5: No bootstrap required, opened a new log
I20260809 02:14:46.225409 12741 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } }
I20260809 02:14:46.225572 12741 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:46.225596 12741 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd60f1bf463d478ea57308d77ab8b8c5, State: Initialized, Role: FOLLOWER
I20260809 02:14:46.225706 12741 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } }
I20260809 02:14:46.225762 12741 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:46.225785 12741 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:46.225863 12741 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:46.226754 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12663
I20260809 02:14:46.226856 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:46.226840 12741 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } }
I20260809 02:14:46.226946 12741 leader_election.cc:302] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd60f1bf463d478ea57308d77ab8b8c5; no voters:
I20260809 02:14:46.227054 9377 external_mini_cluster.cc:1428] Setting key e9112b9eb1386a062a7d80e12610fc9e
I20260809 02:14:46.227176 12741 leader_election.cc:288] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:46.227234 12745 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:46.227422 12745 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [term 1 LEADER]: Becoming Leader. State: Replica: bd60f1bf463d478ea57308d77ab8b8c5, State: Running, Role: LEADER
I20260809 02:14:46.227459 12741 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:46.227526 12745 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } }
I20260809 02:14:46.227862 12745 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd60f1bf463d478ea57308d77ab8b8c5. Latest consensus state: current_term: 1 leader_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } } }
I20260809 02:14:46.227949 12745 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:46.227890 12747 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd60f1bf463d478ea57308d77ab8b8c5" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45217 } } }
I20260809 02:14:46.228096 12747 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:46.228495 12754 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:46.229130 12754 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:46.230787 12754 catalog_manager.cc:1347] Generated new cluster ID: b01f91aab7f14f6d806fc3a14b4f34c9
I20260809 02:14:46.230837 12754 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12648](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241686, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:46.238533 12754 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:46.239274 12754 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:46.239961 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:46.227730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52132 (local address 127.9.40.126:45217)
0809 02:14:46.227918 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:46.227929 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:46.228157 (+ 228us) server_negotiation.cc:407] Connection header received
0809 02:14:46.228298 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:46.228305 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:46.228367 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:46.229614 (+ 1247us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:46.229728 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:46.230852 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:46.231963 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:46.232610 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:46.232716 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:46.235657 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:46.235662 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:46.235662 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:46.235676 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:46.237934 (+ 2258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:46.238603 (+ 669us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:46.238607 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:46.238607 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:46.238731 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:46.238952 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:46.238953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:46.238954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:46.239236 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:46.239420 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:46.239610 (+ 190us) server_negotiation.cc:299] Negotiation successful
0809 02:14:46.239777 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
I20260809 02:14:46.240975 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:46.246165 12754 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd60f1bf463d478ea57308d77ab8b8c5: Generated new TSK 0
I20260809 02:14:46.246372 12754 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:46.257825 12749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:46.253783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52136 (local address 127.9.40.126:45217)
0809 02:14:46.253855 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:46.253857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:46.253998 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:14:46.254042 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:46.254043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:46.254055 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:46.254147 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:46.254494 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:46.255321 (+ 827us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:46.256098 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:46.256455 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:46.256529 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:46.257286 (+ 757us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:46.257287 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:46.257288 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:46.257552 (+ 264us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:46.257572 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:46.257580 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:46.257640 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:46.257706 (+ 66us) server_negotiation.cc:299] Negotiation successful
0809 02:14:46.257736 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:14:46.260699 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12663
2026-08-09T02:14:46Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1428 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12779](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12779](info): set up 2 sockets
Aug 09 02:14:46 dist-test-slave-qg60 krb5kdc[12779](info): commencing operation
krb5kdc: starting...
W20260809 02:14:47.363798 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:47.488603 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37725
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37725
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:47.575244 12793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:47.575564 12793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:47.575614 12793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:47.576946 12793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:47.577026 12793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:47.577056 12793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:47.577090 12793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:47.577117 12793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:47.578788 12793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37725
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37725
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:47.579130 12793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:47.579383 12793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:47.584709 12799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:47.584846 12800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:47.585083 12793 server_base.cc:1034] running on GCE node
W20260809 02:14:47.584761 12802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:47.585444 12793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:47.585757 12793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:47.586997 12793 hybrid_clock.cc:648] HybridClock initialized: now 1786241687586993 us; error 40 us; skew 500 ppm
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:47.589654 12793 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:47.590070 12793 webserver.cc:460] Webserver started at http://127.9.40.126:33081/ using document root <none> and password file <none>
I20260809 02:14:47.590312 12793 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:47.590360 12793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:47.590490 12793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:47.591583 12793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "6aa7eb516aa54bafac74f37c1267bd6a"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "7a503c970a2cea3ac23c323e102aba99"
server_key_iv: "c3e9669df2b112dcc2880cad5a711998"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.591958 12793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6aa7eb516aa54bafac74f37c1267bd6a"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "7a503c970a2cea3ac23c323e102aba99"
server_key_iv: "c3e9669df2b112dcc2880cad5a711998"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.593497 12793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:47.594355 12809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.594558 12793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:47.594666 12793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "6aa7eb516aa54bafac74f37c1267bd6a"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "7a503c970a2cea3ac23c323e102aba99"
server_key_iv: "c3e9669df2b112dcc2880cad5a711998"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.594751 12793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:47.606657 12793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:47.609169 12793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:47.609340 12793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:47.613534 12793 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37725
I20260809 02:14:47.613583 12868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37725 every 8 connection(s)
I20260809 02:14:47.613969 12793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:47.615058 12869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:47.617583 12869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a: Bootstrap starting.
I20260809 02:14:47.618196 12869 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:47.618521 12869 log.cc:824] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:47.619256 12869 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a: No bootstrap required, opened a new log
I20260809 02:14:47.620988 12869 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } }
I20260809 02:14:47.621141 12869 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:47.621166 12869 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6aa7eb516aa54bafac74f37c1267bd6a, State: Initialized, Role: FOLLOWER
I20260809 02:14:47.621349 12869 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } }
I20260809 02:14:47.621433 12869 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:47.621456 12869 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:47.621501 12869 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:47.622393 12869 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } }
I20260809 02:14:47.622483 12869 leader_election.cc:302] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6aa7eb516aa54bafac74f37c1267bd6a; no voters:
I20260809 02:14:47.622689 12869 leader_election.cc:288] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:47.622735 12872 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:47.622936 12872 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [term 1 LEADER]: Becoming Leader. State: Replica: 6aa7eb516aa54bafac74f37c1267bd6a, State: Running, Role: LEADER
I20260809 02:14:47.622960 12869 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:47.623042 12872 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } }
I20260809 02:14:47.623409 12873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } } }
I20260809 02:14:47.623514 12873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:47.623420 12874 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6aa7eb516aa54bafac74f37c1267bd6a. Latest consensus state: current_term: 1 leader_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6aa7eb516aa54bafac74f37c1267bd6a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37725 } } }
I20260809 02:14:47.623756 12874 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:47.623759 12884 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:47.624008 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12793
I20260809 02:14:47.624104 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:47.624305 9377 external_mini_cluster.cc:1428] Setting key 507a16bd2006c010e81618143a0090b3
I20260809 02:14:47.624521 12884 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:47.626291 12884 catalog_manager.cc:1347] Generated new cluster ID: a92c4cd1237e4131b3463467abc6bff8
I20260809 02:14:47.626350 12884 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:47.634964 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:47.624999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42556 (local address 127.9.40.126:37725)
0809 02:14:47.625178 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:47.625187 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:47.625725 (+ 538us) server_negotiation.cc:407] Connection header received
0809 02:14:47.625863 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:47.625871 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:47.625930 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:47.626070 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:47.626813 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.627745 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.628526 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.629073 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.629190 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:47.631571 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:47.631575 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:47.631575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:47.631592 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:47.633369 (+ 1777us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:47.633769 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:47.633772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:47.633773 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:47.633880 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:47.634088 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:47.634089 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:47.634089 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:47.634338 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:47.634520 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:47.634596 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 02:14:47.634748 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260809 02:14:47.653101 12884 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:47.653867 12884 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:47.660116 12884 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6aa7eb516aa54bafac74f37c1267bd6a: Generated new TSK 0
I20260809 02:14:47.660326 12884 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:47.696978 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:47.793006 12899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:47.793282 12899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:47.793311 12899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:47.794557 12899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:47.794615 12899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:47.794644 12899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:47.796240 12899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:47.796452 12899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:47.796808 12899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:47.802086 12905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:47.802130 12907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:47.802130 12904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:47.802435 12899 server_base.cc:1034] running on GCE node
I20260809 02:14:47.802693 12899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:47.802950 12899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:47.804118 12899 hybrid_clock.cc:648] HybridClock initialized: now 1786241687804147 us; error 85 us; skew 500 ppm
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:47.807024 12899 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:47.807550 12899 webserver.cc:460] Webserver started at http://127.9.40.65:33663/ using document root <none> and password file <none>
I20260809 02:14:47.807797 12899 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:47.807869 12899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:47.808029 12899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:47.809302 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "9e150e8ac170483d9627b313dc247768"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "29d368436919a1ae69c19dac346ce46c"
server_key_iv: "1004c0ed2ad6aa0881f8a514774092cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.809682 12899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "9e150e8ac170483d9627b313dc247768"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "29d368436919a1ae69c19dac346ce46c"
server_key_iv: "1004c0ed2ad6aa0881f8a514774092cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.811065 12899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:47.811836 12914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.812011 12899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:47.812114 12899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "9e150e8ac170483d9627b313dc247768"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "29d368436919a1ae69c19dac346ce46c"
server_key_iv: "1004c0ed2ad6aa0881f8a514774092cc"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.812206 12899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:47.821532 12899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:47.824162 12899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:47.824330 12899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:47.824581 12899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:47.824978 12899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:47.825035 12899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.825070 12899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:47.825100 12899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.831260 12899 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38883
I20260809 02:14:47.831367 13038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38883 every 8 connection(s)
I20260809 02:14:47.831683 12899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:47.833894 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 12899
I20260809 02:14:47.833976 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:47.834210 9377 external_mini_cluster.cc:1428] Setting key 03f9426943338b8443ebb7861e46ce46
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:47.843236 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:47.832464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42577 (local address 127.9.40.126:37725)
0809 02:14:47.832540 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:47.832543 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:47.832791 (+ 248us) server_negotiation.cc:407] Connection header received
0809 02:14:47.833301 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:47.833302 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:47.833314 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:47.833380 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:47.834457 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.835147 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.836716 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.837143 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.837230 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:47.839961 (+ 2731us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:47.839963 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:47.839964 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:47.839972 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:47.841573 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:47.842157 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:47.842158 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:47.842158 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:47.842214 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:47.842518 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:47.842519 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:47.842519 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:47.842632 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:47.842686 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:47.843063 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 02:14:47.843164 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:14:47.843549 13042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:47.832427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37725 (local address 127.9.40.65:42577)
0809 02:14:47.832592 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:47.832637 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:47.833203 (+ 566us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:47.833432 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:47.833436 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:47.834101 (+ 665us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:47.834361 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:47.834367 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.835259 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.835260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:47.836590 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:47.836593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.837246 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.837247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:47.837357 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:47.837941 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:47.837950 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:47.839833 (+ 1883us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:47.841671 (+ 1838us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:47.841681 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:47.841683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:47.842048 (+ 365us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:47.842281 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:47.842282 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:47.842282 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:47.842445 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:47.842703 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:47.842709 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:47.842900 (+ 191us) client_negotiation.cc:769] Sending connection context
0809 02:14:47.843052 (+ 152us) client_negotiation.cc:240] Negotiation successful
0809 02:14:47.843253 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260809 02:14:47.844175 13039 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37725
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:14:47.844281 13039 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:47.844491 13039 heartbeater.cc:511] Master 127.9.40.126:37725 requested a full tablet report, sending...
I20260809 02:14:47.845095 12828 ts_manager.cc:194] Registered new tserver with Master: 9e150e8ac170483d9627b313dc247768 (127.9.40.65:38883)
I20260809 02:14:47.845774 12828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:42577
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:47.867758 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:47.953912 13045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:47.954192 13045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:47.954236 13045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:47.955474 13045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:47.955549 13045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:47.955603 13045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:47.957201 13045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:47.957484 13045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:47.957741 13045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:47.963016 13053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:47.963137 13045 server_base.cc:1034] running on GCE node
W20260809 02:14:47.963055 13050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:47.963018 13051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:47.963500 13045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:47.963785 13045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:47.964954 13045 hybrid_clock.cc:648] HybridClock initialized: now 1786241687964947 us; error 35 us; skew 500 ppm
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:47.967617 13045 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:47.968087 13045 webserver.cc:460] Webserver started at http://127.9.40.66:36347/ using document root <none> and password file <none>
I20260809 02:14:47.968377 13045 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:47.968449 13045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:47.968605 13045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:47.969825 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "a9c1563487d24b8792d640739758a4c5"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "95899a8fa520f43220d1951851d0f00b"
server_key_iv: "a3537a11660869d9132d7e52419ae9c6"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.970204 13045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "a9c1563487d24b8792d640739758a4c5"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "95899a8fa520f43220d1951851d0f00b"
server_key_iv: "a3537a11660869d9132d7e52419ae9c6"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.971583 13045 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:47.972448 13062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.972677 13045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:47.972795 13045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "a9c1563487d24b8792d640739758a4c5"
format_stamp: "Formatted at 2026-08-09 02:14:47 on dist-test-slave-qg60"
server_key: "95899a8fa520f43220d1951851d0f00b"
server_key_iv: "a3537a11660869d9132d7e52419ae9c6"
server_key_version: "encryptionkey@0"
I20260809 02:14:47.972884 13045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:47.984927 13045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:47.987105 13045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:47.987279 13045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:47.987532 13045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:47.987871 13045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:47.987929 13045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.987977 13045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:47.988009 13045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:47.994367 13045 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:37675
I20260809 02:14:47.994415 13193 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:37675 every 8 connection(s)
I20260809 02:14:47.994747 13045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:14:47 dist-test-slave-qg60 krb5kdc[12779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241687, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:48.004040 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13045
I20260809 02:14:48.004145 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:48.004375 9377 external_mini_cluster.cc:1428] Setting key bfa3b0a58f0ade180afbbf327bfada21
I20260809 02:14:48.006109 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:47.995608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35001 (local address 127.9.40.126:37725)
0809 02:14:47.995713 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:47.995716 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:47.995896 (+ 180us) server_negotiation.cc:407] Connection header received
0809 02:14:47.996515 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:47.996517 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:47.996530 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:47.996597 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:47.997792 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.998376 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.999552 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.999794 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.999889 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:48.002604 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:48.002606 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:48.002607 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:48.002616 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:48.004349 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.004961 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.004963 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.004963 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.005035 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.005335 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.005336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.005337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.005466 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:48.005581 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:48.005896 (+ 315us) server_negotiation.cc:299] Negotiation successful
0809 02:14:48.005998 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260809 02:14:48.006361 13197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:47.995513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37725 (local address 127.9.40.66:35001)
0809 02:14:47.995742 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:47.995796 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:47.996388 (+ 592us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:47.996685 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:47.996688 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:47.997396 (+ 708us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:47.997688 (+ 292us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:47.997694 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.998481 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.998482 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:47.999421 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:47.999423 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:47.999904 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:47.999905 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:48.000057 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:48.000702 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:48.000711 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:48.002451 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:48.004449 (+ 1998us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:48.004459 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:48.004463 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:48.004852 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:48.005111 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:48.005112 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:48.005113 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:48.005252 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:48.005557 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:48.005565 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:48.005741 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 02:14:48.005858 (+ 117us) client_negotiation.cc:240] Negotiation successful
0809 02:14:48.006076 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":121,"threads_started":1}
I20260809 02:14:48.006947 13195 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37725
I20260809 02:14:48.007089 13195 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:48.007297 13195 heartbeater.cc:511] Master 127.9.40.126:37725 requested a full tablet report, sending...
I20260809 02:14:48.007735 12828 ts_manager.cc:194] Registered new tserver with Master: a9c1563487d24b8792d640739758a4c5 (127.9.40.66:37675)
I20260809 02:14:48.008165 12828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35001
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:48.031863 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:48.116371 13200 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:48.116597 13200 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:48.116622 13200 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:48.117847 13200 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:48.117895 13200 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:48.117923 13200 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:48.119494 13200 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37725
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13200
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:48.119764 13200 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:48.120007 13200 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:48.125461 13206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:48.125535 13200 server_base.cc:1034] running on GCE node
W20260809 02:14:48.125475 13210 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:48.125694 13207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:48.125969 13200 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:48.126267 13200 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:48.127441 13200 hybrid_clock.cc:648] HybridClock initialized: now 1786241688127429 us; error 31 us; skew 500 ppm
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241688, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:48.130215 13200 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:48.130781 13200 webserver.cc:460] Webserver started at http://127.9.40.67:38155/ using document root <none> and password file <none>
I20260809 02:14:48.131042 13200 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:48.131111 13200 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:48.131268 13200 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:48.132431 13200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "301711d3001145c48d3d33bb8532eb4c"
format_stamp: "Formatted at 2026-08-09 02:14:48 on dist-test-slave-qg60"
server_key: "70c4470210ad6dd80e0aca25b1e55977"
server_key_iv: "f3a789bd81ccf60ddcc1d9afdb4be547"
server_key_version: "encryptionkey@0"
I20260809 02:14:48.132901 13200 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "301711d3001145c48d3d33bb8532eb4c"
format_stamp: "Formatted at 2026-08-09 02:14:48 on dist-test-slave-qg60"
server_key: "70c4470210ad6dd80e0aca25b1e55977"
server_key_iv: "f3a789bd81ccf60ddcc1d9afdb4be547"
server_key_version: "encryptionkey@0"
I20260809 02:14:48.134414 13200 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:48.135352 13217 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:48.135507 13200 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:48.135597 13200 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "301711d3001145c48d3d33bb8532eb4c"
format_stamp: "Formatted at 2026-08-09 02:14:48 on dist-test-slave-qg60"
server_key: "70c4470210ad6dd80e0aca25b1e55977"
server_key_iv: "f3a789bd81ccf60ddcc1d9afdb4be547"
server_key_version: "encryptionkey@0"
I20260809 02:14:48.135679 13200 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:48.154932 13200 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:48.157546 13200 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:48.157680 13200 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:48.157883 13200 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:48.158212 13200 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:48.158252 13200 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:48.158277 13200 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:48.158289 13200 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:48.164772 13200 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:41063
I20260809 02:14:48.164891 13340 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:41063 every 8 connection(s)
I20260809 02:14:48.165225 13200 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:48.167618 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13200
I20260809 02:14:48.167749 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:48.167969 9377 external_mini_cluster.cc:1428] Setting key 5aee6d283a8747f22420e00f9bcf735d
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[12779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241688, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:48.175421 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:48.166028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:34067 (local address 127.9.40.126:37725)
0809 02:14:48.166110 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:48.166112 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:48.166266 (+ 154us) server_negotiation.cc:407] Connection header received
0809 02:14:48.166786 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:48.166787 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:48.166798 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:48.166870 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:48.167857 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.168703 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.169842 (+ 1139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.170057 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.170128 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:48.172555 (+ 2427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:48.172557 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:48.172557 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:48.172565 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:48.174098 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.174508 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.174509 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.174509 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.174566 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.174815 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.174815 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.174816 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.174933 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:48.174998 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:48.175264 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:14:48.175346 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260809 02:14:48.175570 13343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:48.165982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37725 (local address 127.9.40.67:34067)
0809 02:14:48.166147 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:48.166192 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:48.166690 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:48.166924 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:48.166927 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:48.167525 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:48.167757 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:48.167764 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.168839 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.168840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:48.169759 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:48.169762 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.170161 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.170162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:48.170286 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:48.170898 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:48.170907 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:48.172433 (+ 1526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:48.174201 (+ 1768us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:48.174207 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:48.174209 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:48.174436 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:48.174627 (+ 191us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:48.174627 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:48.174628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:48.174735 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:48.174995 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:48.175001 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:48.175137 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 02:14:48.175225 (+ 88us) client_negotiation.cc:240] Negotiation successful
0809 02:14:48.175352 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":94,"threads_started":1}
I20260809 02:14:48.176246 13341 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37725
I20260809 02:14:48.176352 13341 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:48.176520 13341 heartbeater.cc:511] Master 127.9.40.126:37725 requested a full tablet report, sending...
I20260809 02:14:48.176993 12828 ts_manager.cc:194] Registered new tserver with Master: 301711d3001145c48d3d33bb8532eb4c (127.9.40.67:41063)
I20260809 02:14:48.177381 12828 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:34067
I20260809 02:14:48.180037 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[12779](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241688, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[12779](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241688, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:48.206578 12893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:48.198725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42562 (local address 127.9.40.126:37725)
0809 02:14:48.198802 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:48.198805 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:48.198986 (+ 181us) server_negotiation.cc:407] Connection header received
0809 02:14:48.199052 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:48.199053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:48.199066 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:48.199117 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:48.199707 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.200287 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.201124 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:48.201331 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:48.201395 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:48.203919 (+ 2524us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:48.203921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:48.203921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:48.203928 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:48.205398 (+ 1470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.205748 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.205749 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.205749 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.205803 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:48.205974 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:48.205975 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:48.205975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:48.206205 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:48.206295 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:48.206387 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 02:14:48.206454 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 02:14:48.206876 12828 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:42562
I20260809 02:14:48.207422 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12899
I20260809 02:14:48.214710 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13045
I20260809 02:14:48.221043 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13200
I20260809 02:14:48.228645 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 12793
2026-08-09T02:14:48Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1968 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[13354](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[13354](info): set up 2 sockets
Aug 09 02:14:48 dist-test-slave-qg60 krb5kdc[13354](info): commencing operation
krb5kdc: starting...
W20260809 02:14:49.356464 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.105s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:49.487190 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41139
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41139
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:49.569757 13369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:49.569974 13369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:49.569998 13369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:49.571182 13369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:49.571228 13369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:49.571241 13369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:49.571254 13369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:49.571310 13369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:49.572947 13369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41139
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41139
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:49.573215 13369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:49.573446 13369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:49.578920 13369 server_base.cc:1034] running on GCE node
W20260809 02:14:49.578967 13377 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.578864 13374 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.578882 13375 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:49.579447 13369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:49.579790 13369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:49.580960 13369 hybrid_clock.cc:648] HybridClock initialized: now 1786241689580944 us; error 44 us; skew 500 ppm
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:49.584017 13369 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:49.584590 13369 webserver.cc:460] Webserver started at http://127.9.40.126:41719/ using document root <none> and password file <none>
I20260809 02:14:49.584875 13369 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:49.584936 13369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:49.585098 13369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:49.586242 13369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "88b10188b4414b9890d183351d1fd7fe"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "b2ce84c56f18e067beb0f3f845abf01c"
server_key_iv: "d12604622549ec8614d041e1b4cd33d4"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.586620 13369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "88b10188b4414b9890d183351d1fd7fe"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "b2ce84c56f18e067beb0f3f845abf01c"
server_key_iv: "d12604622549ec8614d041e1b4cd33d4"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.587985 13369 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:49.588773 13384 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.588974 13369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:49.589076 13369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "88b10188b4414b9890d183351d1fd7fe"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "b2ce84c56f18e067beb0f3f845abf01c"
server_key_iv: "d12604622549ec8614d041e1b4cd33d4"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.589218 13369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:49.604700 13369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:49.606976 13369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:49.607149 13369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:49.611410 13369 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41139
I20260809 02:14:49.611464 13444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41139 every 8 connection(s)
I20260809 02:14:49.612039 13369 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:49.612721 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13369
I20260809 02:14:49.612859 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:49.612933 13445 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:49.613086 9377 external_mini_cluster.cc:1428] Setting key 98e4aeef4532ca4d949ad9d26f81da36
I20260809 02:14:49.616228 13445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe: Bootstrap starting.
I20260809 02:14:49.617027 13445 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:49.617319 13445 log.cc:824] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:49.618108 13445 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe: No bootstrap required, opened a new log
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:49.619887 13445 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } }
I20260809 02:14:49.620054 13445 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:49.620105 13445 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88b10188b4414b9890d183351d1fd7fe, State: Initialized, Role: FOLLOWER
I20260809 02:14:49.620245 13445 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } }
I20260809 02:14:49.620311 13445 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:49.620332 13445 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:49.620402 13445 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:49.621234 13445 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } }
I20260809 02:14:49.621361 13445 leader_election.cc:302] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88b10188b4414b9890d183351d1fd7fe; no voters:
I20260809 02:14:49.621542 13445 leader_election.cc:288] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:49.621583 13452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:49.621752 13452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [term 1 LEADER]: Becoming Leader. State: Replica: 88b10188b4414b9890d183351d1fd7fe, State: Running, Role: LEADER
I20260809 02:14:49.621834 13445 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:49.621872 13452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } }
I20260809 02:14:49.622246 13454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88b10188b4414b9890d183351d1fd7fe. Latest consensus state: current_term: 1 leader_uuid: "88b10188b4414b9890d183351d1fd7fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } } }
I20260809 02:14:49.622355 13454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:49.622251 13453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88b10188b4414b9890d183351d1fd7fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88b10188b4414b9890d183351d1fd7fe" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41139 } } }
I20260809 02:14:49.622655 13453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:49.623533 13459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:49.624344 13459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:49.625373 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:49.613675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59610 (local address 127.9.40.126:41139)
0809 02:14:49.613869 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:49.613885 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:49.613912 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:14:49.614467 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:49.614474 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:49.614530 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:49.614652 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:49.615458 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.616400 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.617481 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.618014 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.618129 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:49.620611 (+ 2482us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:49.620615 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:49.620615 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:49.620627 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:49.623561 (+ 2934us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.624024 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.624027 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.624027 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.624148 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.624389 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.624391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.624391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.624701 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:49.624900 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:49.625027 (+ 127us) server_negotiation.cc:299] Negotiation successful
0809 02:14:49.625180 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260809 02:14:49.625995 13459 catalog_manager.cc:1347] Generated new cluster ID: b2827b4af5104432ad2b7504fd8d9ee1
I20260809 02:14:49.626052 13459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:49.629837 13459 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:49.630548 13459 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:49.641528 13459 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88b10188b4414b9890d183351d1fd7fe: Generated new TSK 0
I20260809 02:14:49.641757 13459 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:49.675817 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:49.761175 13476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:49.761373 13476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:49.761397 13476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:49.762490 13476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:49.762524 13476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:49.762549 13476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:49.763986 13476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:49.764170 13476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:49.764355 13476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:49.769351 13484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.769333 13481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.769321 13482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:49.769626 13476 server_base.cc:1034] running on GCE node
I20260809 02:14:49.769827 13476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:49.770151 13476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:49.771322 13476 hybrid_clock.cc:648] HybridClock initialized: now 1786241689771303 us; error 35 us; skew 500 ppm
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:49.774144 13476 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:49.774628 13476 webserver.cc:460] Webserver started at http://127.9.40.65:36145/ using document root <none> and password file <none>
I20260809 02:14:49.774912 13476 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:49.774993 13476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:49.775141 13476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:49.776265 13476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "b97a919dc55043c59df1693e19b4c12b"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "00259c194918d9e4c5395fb254185b97"
server_key_iv: "02a07b75c6f29c3f18b0bd25d9c55dbe"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.776641 13476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "b97a919dc55043c59df1693e19b4c12b"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "00259c194918d9e4c5395fb254185b97"
server_key_iv: "02a07b75c6f29c3f18b0bd25d9c55dbe"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.778115 13476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:49.778926 13491 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.779167 13476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:49.779377 13476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "b97a919dc55043c59df1693e19b4c12b"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "00259c194918d9e4c5395fb254185b97"
server_key_iv: "02a07b75c6f29c3f18b0bd25d9c55dbe"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.779459 13476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:49.790424 13476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:49.792691 13476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:49.792903 13476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:49.793212 13476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:49.793605 13476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:49.793642 13476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.793666 13476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:49.793679 13476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.799666 13476 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:36385
I20260809 02:14:49.799763 13616 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:36385 every 8 connection(s)
I20260809 02:14:49.800089 13476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:49.801275 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13476
I20260809 02:14:49.801394 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:49.801585 9377 external_mini_cluster.cc:1428] Setting key 2a0fb6336332f3ceef1375987e3271bd
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:14:49.811667 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:49.800926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36365 (local address 127.9.40.126:41139)
0809 02:14:49.801012 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:49.801014 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:49.801308 (+ 294us) server_negotiation.cc:407] Connection header received
0809 02:14:49.802717 (+ 1409us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:49.802719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:49.802732 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:49.802807 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:49.803418 (+ 611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.804210 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.805888 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.806106 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.806173 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:49.808755 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:49.808757 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:49.808757 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:49.808766 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:49.810206 (+ 1440us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.810617 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.810619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.810619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.810683 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.810941 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.810942 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.810942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.811059 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:49.811135 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:49.811453 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 02:14:49.811550 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:14:49.811825 13619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:49.800896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41139 (local address 127.9.40.65:36365)
0809 02:14:49.801143 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:49.801198 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:49.802607 (+ 1409us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:49.802864 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:49.802867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:49.803133 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:49.803326 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:49.803330 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.804324 (+ 994us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.804325 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:49.805704 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:49.805707 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.806204 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.806205 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:49.806358 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:49.806932 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:49.806944 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:49.808625 (+ 1681us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:49.810296 (+ 1671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:49.810304 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:49.810307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:49.810539 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:49.810753 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:49.810754 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:49.810754 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:49.810867 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:49.811125 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:49.811153 (+ 28us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:49.811310 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 02:14:49.811415 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:14:49.811584 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":106,"threads_started":1}
I20260809 02:14:49.812379 13617 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41139
I20260809 02:14:49.812487 13617 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:49.812695 13617 heartbeater.cc:511] Master 127.9.40.126:41139 requested a full tablet report, sending...
I20260809 02:14:49.813292 13407 ts_manager.cc:194] Registered new tserver with Master: b97a919dc55043c59df1693e19b4c12b (127.9.40.65:36385)
I20260809 02:14:49.813954 13407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36365
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:49.831305 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:49.915632 13623 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:49.915944 13623 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:49.915995 13623 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:49.917296 13623 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:49.917354 13623 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:49.917404 13623 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:49.919004 13623 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13623
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:49.919265 13623 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:49.919503 13623 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:49.924937 13623 server_base.cc:1034] running on GCE node
W20260809 02:14:49.924919 13632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.924919 13629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:49.924901 13630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:49.925273 13623 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:49.925480 13623 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:49.926623 13623 hybrid_clock.cc:648] HybridClock initialized: now 1786241689926611 us; error 29 us; skew 500 ppm
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:49.929425 13623 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:49.929921 13623 webserver.cc:460] Webserver started at http://127.9.40.66:35387/ using document root <none> and password file <none>
I20260809 02:14:49.930171 13623 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:49.930241 13623 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:49.930403 13623 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:49.931560 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "b287f64b6b914a979a18e79b1ed011c0"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "532100a1d735bf0bf1ad16f749d47880"
server_key_iv: "6ebf558caa3a533a783bc821abb3748a"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.931938 13623 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "b287f64b6b914a979a18e79b1ed011c0"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "532100a1d735bf0bf1ad16f749d47880"
server_key_iv: "6ebf558caa3a533a783bc821abb3748a"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.933487 13623 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:49.934230 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.934399 13623 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:49.934506 13623 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "b287f64b6b914a979a18e79b1ed011c0"
format_stamp: "Formatted at 2026-08-09 02:14:49 on dist-test-slave-qg60"
server_key: "532100a1d735bf0bf1ad16f749d47880"
server_key_iv: "6ebf558caa3a533a783bc821abb3748a"
server_key_version: "encryptionkey@0"
I20260809 02:14:49.934590 13623 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:49.945212 13623 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:49.947570 13623 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:49.947736 13623 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:49.947978 13623 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:49.948293 13623 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:49.948346 13623 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.948392 13623 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:49.948415 13623 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:49.954692 13623 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:38097
I20260809 02:14:49.954749 13763 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:38097 every 8 connection(s)
I20260809 02:14:49.955070 13623 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:49.956714 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13623
I20260809 02:14:49.956893 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:49.957073 9377 external_mini_cluster.cc:1428] Setting key 790b2a8bfd1f9521db873cdd63fe52aa
Aug 09 02:14:49 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241689, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260809 02:14:49.965401 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:49.955736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57323 (local address 127.9.40.126:41139)
0809 02:14:49.955817 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:49.955819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:49.956066 (+ 247us) server_negotiation.cc:407] Connection header received
0809 02:14:49.957164 (+ 1098us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:49.957165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:49.957178 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:49.957228 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:49.958058 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.958565 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.959609 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.959798 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.959875 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:49.962415 (+ 2540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:49.962416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:49.962417 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:49.962426 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:49.963874 (+ 1448us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.964310 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.964311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.964311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.964374 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:49.964666 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:49.964668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:49.964668 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:49.964854 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:49.964928 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:49.965219 (+ 291us) server_negotiation.cc:299] Negotiation successful
0809 02:14:49.965306 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:14:49.965598 13767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:49.955739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41139 (local address 127.9.40.66:57323)
0809 02:14:49.955936 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:49.955973 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:49.957052 (+ 1079us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:49.957318 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:49.957322 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:49.957761 (+ 439us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:49.957962 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:49.957967 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.958666 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.958667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:49.959491 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:49.959494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:49.959884 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:49.959885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:49.960014 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:49.960563 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:49.960571 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:49.962317 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:49.963981 (+ 1664us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:49.963989 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:49.963991 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:49.964230 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:49.964451 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:49.964452 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:49.964452 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:49.964574 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:49.964930 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:49.964938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:49.965082 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 02:14:49.965187 (+ 105us) client_negotiation.cc:240] Negotiation successful
0809 02:14:49.965356 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":90,"threads_started":1}
I20260809 02:14:49.966391 13764 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41139
I20260809 02:14:49.966496 13764 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:49.966694 13764 heartbeater.cc:511] Master 127.9.40.126:41139 requested a full tablet report, sending...
I20260809 02:14:49.967197 13407 ts_manager.cc:194] Registered new tserver with Master: b287f64b6b914a979a18e79b1ed011c0 (127.9.40.66:38097)
I20260809 02:14:49.967571 13407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:57323
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:49.984520 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43267
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:50.067461 13771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:50.067677 13771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:50.067700 13771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:50.068881 13771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:50.068934 13771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:50.068986 13771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:50.070508 13771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43267
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41139
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:50.070777 13771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:50.071008 13771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:50.076006 13779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:50.076010 13777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:50.076009 13776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:50.076105 13771 server_base.cc:1034] running on GCE node
I20260809 02:14:50.076526 13771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:50.076771 13771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:50.078027 13771 hybrid_clock.cc:648] HybridClock initialized: now 1786241690078012 us; error 49 us; skew 500 ppm
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241690, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:50.080801 13771 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:50.081272 13771 webserver.cc:460] Webserver started at http://127.9.40.67:45283/ using document root <none> and password file <none>
I20260809 02:14:50.081501 13771 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:50.081573 13771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:50.081727 13771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:50.082886 13771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "9d50a737a3554ccfa5e42e2086f3649e"
format_stamp: "Formatted at 2026-08-09 02:14:50 on dist-test-slave-qg60"
server_key: "2d2f9d4e449887f4cc938422349a56a4"
server_key_iv: "acd38da2b872868bdcd42f4008ad3400"
server_key_version: "encryptionkey@0"
I20260809 02:14:50.083305 13771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "9d50a737a3554ccfa5e42e2086f3649e"
format_stamp: "Formatted at 2026-08-09 02:14:50 on dist-test-slave-qg60"
server_key: "2d2f9d4e449887f4cc938422349a56a4"
server_key_iv: "acd38da2b872868bdcd42f4008ad3400"
server_key_version: "encryptionkey@0"
I20260809 02:14:50.084626 13771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:14:50.085459 13787 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:50.085632 13771 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:14:50.085739 13771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "9d50a737a3554ccfa5e42e2086f3649e"
format_stamp: "Formatted at 2026-08-09 02:14:50 on dist-test-slave-qg60"
server_key: "2d2f9d4e449887f4cc938422349a56a4"
server_key_iv: "acd38da2b872868bdcd42f4008ad3400"
server_key_version: "encryptionkey@0"
I20260809 02:14:50.085817 13771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:50.101193 13771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:50.103724 13771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:50.103904 13771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:50.104192 13771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:50.104526 13771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:50.104583 13771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:50.104629 13771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:50.104653 13771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:50.111065 13771 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:46341
I20260809 02:14:50.111146 13911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:46341 every 8 connection(s)
I20260809 02:14:50.111469 13771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241690, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:50.120278 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13771
I20260809 02:14:50.120404 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:50.120684 9377 external_mini_cluster.cc:1428] Setting key 0705b7646eb2addee6b9ae081eb07c8e
I20260809 02:14:50.121304 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:50.112250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49453 (local address 127.9.40.126:41139)
0809 02:14:50.112313 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:50.112315 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:50.112504 (+ 189us) server_negotiation.cc:407] Connection header received
0809 02:14:50.113551 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:50.113552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:50.113564 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:50.113632 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:50.114291 (+ 659us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.114816 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.115807 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.116005 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.116069 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:50.118277 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:50.118278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:50.118278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:50.118285 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:50.119729 (+ 1444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.120106 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.120107 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.120108 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.120160 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.120468 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.120469 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.120470 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.120597 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:50.120651 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:50.121058 (+ 407us) server_negotiation.cc:299] Negotiation successful
0809 02:14:50.121214 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:14:50.121441 13914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:50.112173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41139 (local address 127.9.40.67:49453)
0809 02:14:50.112376 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:50.112417 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:50.113428 (+ 1011us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:50.113713 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:50.113716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:50.113981 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:50.114194 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:50.114198 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.114899 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.114899 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:50.115693 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:50.115695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.116085 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.116086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:50.116201 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:50.116667 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:50.116675 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:50.118187 (+ 1512us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:50.119802 (+ 1615us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:50.119807 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:50.119809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:50.120017 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:50.120232 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:50.120233 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:50.120233 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:50.120386 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:50.120678 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:50.120685 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:50.120888 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:14:50.121029 (+ 141us) client_negotiation.cc:240] Negotiation successful
0809 02:14:50.121206 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 02:14:50.121833 13912 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41139
I20260809 02:14:50.121937 13912 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:50.122083 13912 heartbeater.cc:511] Master 127.9.40.126:41139 requested a full tablet report, sending...
I20260809 02:14:50.122519 13407 ts_manager.cc:194] Registered new tserver with Master: 9d50a737a3554ccfa5e42e2086f3649e (127.9.40.67:46341)
I20260809 02:14:50.122956 13407 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:49453
I20260809 02:14:50.132908 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13354](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:50.157863 13919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:50.149776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56944 (local address 127.9.40.65:36385)
0809 02:14:50.149957 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:50.149967 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:50.149979 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:50.150008 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:50.150010 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:50.150030 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:50.150097 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:50.150707 (+ 610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.151497 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.152194 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.152496 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.152559 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:50.154835 (+ 2276us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:50.154843 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:50.154843 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:50.154850 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:50.156529 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.156935 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.156938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.156939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.157007 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.157227 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.157227 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.157228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.157356 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:50.157462 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:50.157518 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:14:50.157642 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":104,"threads_started":1}
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13354](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241690, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:50.168639 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:50.159597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59618 (local address 127.9.40.126:41139)
0809 02:14:50.159679 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:50.159681 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:50.159699 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:14:50.159755 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:50.159756 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:50.159767 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:50.159846 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:50.160523 (+ 677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.161442 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.162509 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:50.162889 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:50.162953 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:50.165821 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:50.165822 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:50.165823 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:50.165829 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:50.167582 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.167954 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.167955 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.167956 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.168011 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:50.168183 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:50.168183 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:50.168184 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:50.168318 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:50.168395 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:50.168464 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:14:50.168532 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
W20260809 02:14:50.168951 13407 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59618
I20260809 02:14:50.169320 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13476
I20260809 02:14:50.176087 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13623
I20260809 02:14:50.181934 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13771
I20260809 02:14:50.188159 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13369
2026-08-09T02:14:50Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (1959 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13924](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13924](info): set up 2 sockets
Aug 09 02:14:50 dist-test-slave-qg60 krb5kdc[13924](info): commencing operation
krb5kdc: starting...
W20260809 02:14:51.276638 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:51.412530 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44949
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:44949
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:51.494612 13939 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:51.494863 13939 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:51.494905 13939 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:51.496120 13939 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:51.496197 13939 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:51.496222 13939 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:51.496244 13939 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:51.496273 13939 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:51.497834 13939 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:44949
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:44949
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13939
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:51.498123 13939 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:51.498371 13939 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:51.503472 13944 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.503512 13945 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.503472 13947 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:51.503816 13939 server_base.cc:1034] running on GCE node
I20260809 02:14:51.504053 13939 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:51.504317 13939 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:51.505517 13939 hybrid_clock.cc:648] HybridClock initialized: now 1786241691505495 us; error 27 us; skew 500 ppm
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:51.513288 13939 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:51.513687 13939 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:51.513765 13939 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:51.513921 13939 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:51.515080 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "f9f9af904dca4f00ac218b68ac0d9b56"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "24575b9b91c00c46e8b53037b32d5ecd"
server_key_iv: "7a106051014482c2d7a9396d4b83e36b"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.515475 13939 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "f9f9af904dca4f00ac218b68ac0d9b56"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "24575b9b91c00c46e8b53037b32d5ecd"
server_key_iv: "7a106051014482c2d7a9396d4b83e36b"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.516860 13939 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:51.517681 13953 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.517861 13939 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:51.517971 13939 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "f9f9af904dca4f00ac218b68ac0d9b56"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "24575b9b91c00c46e8b53037b32d5ecd"
server_key_iv: "7a106051014482c2d7a9396d4b83e36b"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.518047 13939 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:51.536962 13939 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:51.538945 13939 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:51.539126 13939 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:51.543358 13939 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:44949
I20260809 02:14:51.543408 14014 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:44949 every 8 connection(s)
I20260809 02:14:51.543732 13939 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:51.544857 14015 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:51.547199 14015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: Bootstrap starting.
I20260809 02:14:51.547821 14015 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:51.548034 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 13939
I20260809 02:14:51.548153 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:51.548170 14015 log.cc:824] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:51.548346 9377 external_mini_cluster.cc:1428] Setting key 0e7d71b1bbea266cc29f1a1d990774e7
I20260809 02:14:51.549189 14015 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: No bootstrap required, opened a new log
I20260809 02:14:51.551321 14015 raft_consensus.cc:348] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } }
I20260809 02:14:51.551457 14015 raft_consensus.cc:374] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:51.551481 14015 raft_consensus.cc:729] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f9f9af904dca4f00ac218b68ac0d9b56, State: Initialized, Role: FOLLOWER
I20260809 02:14:51.551676 14015 consensus_queue.cc:260] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } }
I20260809 02:14:51.551769 14015 raft_consensus.cc:388] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:51.551801 14015 raft_consensus.cc:482] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:51.551849 14015 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:51.552608 14015 raft_consensus.cc:504] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } }
I20260809 02:14:51.552740 14015 leader_election.cc:302] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f9f9af904dca4f00ac218b68ac0d9b56; no voters:
I20260809 02:14:51.552937 14015 leader_election.cc:288] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:51.552990 14021 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:51.553193 14021 raft_consensus.cc:686] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [term 1 LEADER]: Becoming Leader. State: Replica: f9f9af904dca4f00ac218b68ac0d9b56, State: Running, Role: LEADER
I20260809 02:14:51.553216 14015 sys_catalog.cc:564] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:51.553354 14021 consensus_queue.cc:237] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } }
I20260809 02:14:51.554318 14022 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } } }
I20260809 02:14:51.554419 14022 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:51.554648 14033 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:51.555112 14023 sys_catalog.cc:455] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f9f9af904dca4f00ac218b68ac0d9b56. Latest consensus state: current_term: 1 leader_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f9f9af904dca4f00ac218b68ac0d9b56" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 44949 } } }
I20260809 02:14:51.555258 14033 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:51.555265 14023 sys_catalog.cc:458] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56 [sys.catalog]: This master's current role is: LEADER
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:51.557044 14033 catalog_manager.cc:1347] Generated new cluster ID: 77f370ba39544a70aa6f4d4e9c86d39b
I20260809 02:14:51.557112 14033 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:51.559902 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:51.548926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48694 (local address 127.9.40.126:44949)
0809 02:14:51.549128 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:51.549138 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:51.549172 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:14:51.549672 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:51.549679 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:51.549727 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:51.549875 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:51.550532 (+ 657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.551420 (+ 888us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.552607 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.553276 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.554004 (+ 728us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:51.556363 (+ 2359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:51.556366 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:51.556366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:51.556377 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:51.558272 (+ 1895us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.558663 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.558666 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.558667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.558765 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.558984 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.558985 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.558985 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.559228 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:51.559408 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:51.559477 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:14:51.559734 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":87,"threads_started":1}
I20260809 02:14:51.585760 14033 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:51.586535 14033 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:51.589569 14033 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: Generated new TSK 0
I20260809 02:14:51.589749 14033 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:51.620203 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:51.703195 14044 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:51.703413 14044 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:51.703440 14044 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:51.704650 14044 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:51.704696 14044 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:51.704771 14044 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:51.706450 14044 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14044
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:51.706707 14044 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:51.706974 14044 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:51.712122 14051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.712113 14050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.712113 14054 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:51.712198 14044 server_base.cc:1034] running on GCE node
I20260809 02:14:51.712489 14044 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:51.712795 14044 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:51.713949 14044 hybrid_clock.cc:648] HybridClock initialized: now 1786241691713934 us; error 31 us; skew 500 ppm
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:51.716569 14044 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:51.717020 14044 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:51.717095 14044 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:51.717239 14044 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:51.718317 14044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "1d2ff67c23494901bb94010a01561bde"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "946c9cf360c9e88129e813204e703586"
server_key_iv: "98aa647c23a76aa03d46bce5224a4668"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.718699 14044 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "1d2ff67c23494901bb94010a01561bde"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "946c9cf360c9e88129e813204e703586"
server_key_iv: "98aa647c23a76aa03d46bce5224a4668"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.720108 14044 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:51.720937 14063 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.721131 14044 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:51.721239 14044 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "1d2ff67c23494901bb94010a01561bde"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "946c9cf360c9e88129e813204e703586"
server_key_iv: "98aa647c23a76aa03d46bce5224a4668"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.721330 14044 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:51.736191 14044 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:51.738142 14044 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:51.738319 14044 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:51.738571 14044 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:51.738909 14044 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:51.738965 14044 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.739001 14044 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:51.739051 14044 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.745416 14044 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:43421
I20260809 02:14:51.745461 14188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:43421 every 8 connection(s)
I20260809 02:14:51.745746 14044 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:51.755383 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:51.746511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42097 (local address 127.9.40.126:44949)
0809 02:14:51.746588 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:51.746590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:51.746815 (+ 225us) server_negotiation.cc:407] Connection header received
0809 02:14:51.747350 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:51.747351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:51.747362 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:51.747450 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:51.748441 (+ 991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.749021 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.750037 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.750238 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.750302 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:51.752491 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:51.752492 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:51.752493 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:51.752500 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:51.754002 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.754453 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.754455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.754455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.754514 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.754808 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.754809 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.754809 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.754951 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:51.755005 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:51.755245 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:14:51.755314 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:14:51.755535 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:51.746481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44949 (local address 127.9.40.65:42097)
0809 02:14:51.746672 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:51.746716 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:51.747243 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:51.747514 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:51.747517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:51.747788 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:51.748316 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:51.748321 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.749128 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.749129 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:51.749927 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:51.749929 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.750321 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.750322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:51.750418 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:51.750908 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:51.750916 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:51.752363 (+ 1447us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:51.754083 (+ 1720us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:51.754091 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:51.754093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:51.754379 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:51.754632 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:51.754633 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:51.754633 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:51.754743 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:51.755011 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:51.755016 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:51.755134 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 02:14:51.755217 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:14:51.755339 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":108,"threads_started":1}
I20260809 02:14:51.755966 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14044
I20260809 02:14:51.756091 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:51.756126 14189 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44949
I20260809 02:14:51.756227 14189 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:51.756320 9377 external_mini_cluster.cc:1428] Setting key be46b6d94ae3c2ab03c2390a645a1fac
I20260809 02:14:51.756405 14189 heartbeater.cc:511] Master 127.9.40.126:44949 requested a full tablet report, sending...
I20260809 02:14:51.757025 13969 ts_manager.cc:194] Registered new tserver with Master: 1d2ff67c23494901bb94010a01561bde (127.9.40.65:43421)
I20260809 02:14:51.757895 13969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:42097
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:51.783836 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:51.866727 14196 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:51.866982 14196 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:51.867022 14196 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:51.868188 14196 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:51.868256 14196 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:51.868305 14196 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:51.869910 14196 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14196
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:51.870189 14196 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:51.870424 14196 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:14:51.875564 14196 server_base.cc:1034] running on GCE node
W20260809 02:14:51.875526 14204 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.875448 14201 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:51.875447 14202 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:51.875977 14196 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:51.876183 14196 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:51.877321 14196 hybrid_clock.cc:648] HybridClock initialized: now 1786241691877301 us; error 37 us; skew 500 ppm
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:51.879832 14196 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:51.880213 14196 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:51.880285 14196 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:51.880438 14196 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:51.881547 14196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "f589751780b7480c849cf0f96b427b68"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "c4f372c12ea33faf9d123764aaa59db8"
server_key_iv: "9a0ac03e21622d14430f5404b8985f75"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.881920 14196 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "f589751780b7480c849cf0f96b427b68"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "c4f372c12ea33faf9d123764aaa59db8"
server_key_iv: "9a0ac03e21622d14430f5404b8985f75"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.883208 14196 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:14:51.883936 14211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.884105 14196 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:14:51.884207 14196 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "f589751780b7480c849cf0f96b427b68"
format_stamp: "Formatted at 2026-08-09 02:14:51 on dist-test-slave-qg60"
server_key: "c4f372c12ea33faf9d123764aaa59db8"
server_key_iv: "9a0ac03e21622d14430f5404b8985f75"
server_key_version: "encryptionkey@0"
I20260809 02:14:51.884291 14196 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:51.893740 14196 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:51.895871 14196 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:51.896037 14196 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:51.896286 14196 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:51.896613 14196 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:51.896677 14196 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.896713 14196 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:51.896775 14196 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:51.902915 14196 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:44183
I20260809 02:14:51.902966 14337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:44183 every 8 connection(s)
I20260809 02:14:51.903234 14196 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:14:51 dist-test-slave-qg60 krb5kdc[13924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241691, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:51.909278 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14196
I20260809 02:14:51.909399 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:51.909602 9377 external_mini_cluster.cc:1428] Setting key eed958eb04891585b7381d4e808fb792
I20260809 02:14:51.912384 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:51.903946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59951 (local address 127.9.40.126:44949)
0809 02:14:51.904030 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:51.904033 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:51.904191 (+ 158us) server_negotiation.cc:407] Connection header received
0809 02:14:51.904639 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:51.904640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:51.904651 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:51.904704 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:51.905685 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.906178 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.907133 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.907307 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.907357 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:51.909482 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:51.909484 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:51.909484 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:51.909492 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:51.910992 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.911409 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.911411 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.911411 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.911472 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:51.911733 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:51.911734 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:51.911734 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:51.911854 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:51.911927 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:51.912208 (+ 281us) server_negotiation.cc:299] Negotiation successful
0809 02:14:51.912289 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:14:51.912590 14342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:51.903896 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44949 (local address 127.9.40.66:59951)
0809 02:14:51.904062 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:51.904100 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:51.904558 (+ 458us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:51.904839 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:51.904842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:51.905095 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:51.905581 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:51.905585 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.906264 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.906265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:51.907049 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:51.907051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:51.907381 (+ 330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:51.907382 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:51.907478 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:51.907917 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:51.907924 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:51.909365 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:51.911087 (+ 1722us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:51.911094 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:51.911096 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:51.911317 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:51.911542 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:51.911543 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:51.911544 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:51.911663 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:51.911930 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:51.911938 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:51.912075 (+ 137us) client_negotiation.cc:769] Sending connection context
0809 02:14:51.912184 (+ 109us) client_negotiation.cc:240] Negotiation successful
0809 02:14:51.912339 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260809 02:14:51.913066 14338 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44949
I20260809 02:14:51.913190 14338 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:51.913362 14338 heartbeater.cc:511] Master 127.9.40.126:44949 requested a full tablet report, sending...
I20260809 02:14:51.913772 13969 ts_manager.cc:194] Registered new tserver with Master: f589751780b7480c849cf0f96b427b68 (127.9.40.66:44183)
I20260809 02:14:51.914287 13969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:59951
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:51.939823 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42957
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:52.024845 14346 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:52.025074 14346 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:52.025100 14346 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:52.026273 14346 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:52.026317 14346 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:52.026345 14346 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:52.027849 14346 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42957
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:44949
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14346
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:52.028060 14346 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:52.028259 14346 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:52.033476 14352 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.033476 14354 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.033751 14351 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.033599 14346 server_base.cc:1034] running on GCE node
I20260809 02:14:52.033993 14346 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:52.034373 14346 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:52.035584 14346 hybrid_clock.cc:648] HybridClock initialized: now 1786241692035555 us; error 39 us; skew 500 ppm
Aug 09 02:14:52 dist-test-slave-qg60 krb5kdc[13924](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241692, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:52.038592 14346 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:52.039077 14346 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:52.039165 14346 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:52.039340 14346 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:52.040565 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "63e7824a818b40d9acf2f413cc6f2787"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "deaf8ca0144e54d7e338408f4369bc3d"
server_key_iv: "20b158f52a2f603c5b6967ebf46d4142"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.041009 14346 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "63e7824a818b40d9acf2f413cc6f2787"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "deaf8ca0144e54d7e338408f4369bc3d"
server_key_iv: "20b158f52a2f603c5b6967ebf46d4142"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.042588 14346 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:14:52.043458 14360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.043643 14346 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.043761 14346 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "63e7824a818b40d9acf2f413cc6f2787"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "deaf8ca0144e54d7e338408f4369bc3d"
server_key_iv: "20b158f52a2f603c5b6967ebf46d4142"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.043845 14346 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:52.058467 14346 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:52.061264 14346 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:52.061450 14346 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:52.061739 14346 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:52.062103 14346 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:52.062162 14346 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.062211 14346 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:52.062242 14346 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.068948 14346 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:40533
I20260809 02:14:52.068989 14482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:40533 every 8 connection(s)
I20260809 02:14:52.069281 14346 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:52.075533 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14346
I20260809 02:14:52.075655 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
Aug 09 02:14:52 dist-test-slave-qg60 krb5kdc[13924](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241692, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:52.075860 9377 external_mini_cluster.cc:1428] Setting key f485a68a3e647efdc9126aa569439617
I20260809 02:14:52.079342 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.069968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:49327 (local address 127.9.40.126:44949)
0809 02:14:52.070018 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.070021 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.070263 (+ 242us) server_negotiation.cc:407] Connection header received
0809 02:14:52.070757 (+ 494us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.070758 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.070769 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.070823 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.071859 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.072352 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.073564 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.073753 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.073803 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.076407 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.076409 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.076409 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:52.076419 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.078109 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:52.078478 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:52.078478 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:52.078479 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:52.078530 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:52.078750 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:52.078751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:52.078751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:52.078858 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.078910 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.079187 (+ 277us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.079269 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:14:52.079423 14485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.069978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:44949 (local address 127.9.40.67:49327)
0809 02:14:52.070136 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.070176 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.070670 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.070879 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.070882 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.071167 (+ 285us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.071700 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.071705 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.072447 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.072447 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.073472 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.073475 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.073827 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.073827 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.073931 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.074570 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:52.074578 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.076303 (+ 1725us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.078183 (+ 1880us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:52.078190 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:52.078191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:52.078413 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:52.078586 (+ 173us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:52.078587 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:52.078587 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:52.078697 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:52.078913 (+ 216us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.078918 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.079039 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.079123 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.079251 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":91,"threads_started":1}
I20260809 02:14:52.079948 14483 heartbeater.cc:347] Connected to a master server at 127.9.40.126:44949
I20260809 02:14:52.080118 14483 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:52.080295 14483 heartbeater.cc:511] Master 127.9.40.126:44949 requested a full tablet report, sending...
I20260809 02:14:52.080842 13969 ts_manager.cc:194] Registered new tserver with Master: 63e7824a818b40d9acf2f413cc6f2787 (127.9.40.67:40533)
I20260809 02:14:52.081224 13969 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:49327
I20260809 02:14:52.088181 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:14:52.096171 14020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.089718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48696 (local address 127.9.40.126:44949)
0809 02:14:52.089783 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.089786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.089884 (+ 98us) server_negotiation.cc:407] Connection header received
0809 02:14:52.089965 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.089966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.089977 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.090035 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.090617 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.091114 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.091859 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.092055 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.092129 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.093371 (+ 1242us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.093372 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.093373 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:52.093380 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.094884 (+ 1504us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:52.095233 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:52.095235 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:52.095235 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:52.095293 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:52.095492 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:52.095493 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:52.095493 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:52.095668 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.095778 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.095971 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.096044 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:14:52.097743 13969 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48696:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:52.099300 13969 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:52.105976 14494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.101990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40533 (local address 127.0.0.1:44160)
0809 02:14:52.102356 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.102366 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.102421 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.102695 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.102696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.102698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:52.102780 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.102782 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.103644 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.103645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.104697 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.104699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.105645 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105646 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.105737 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.105756 (+ 19us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.105829 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.105888 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":10,"thread_start_us":86,"threads_started":1}
I20260809 02:14:52.106130 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.101928 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.9.40.65:43421)
0809 02:14:52.102114 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.102119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.102199 (+ 80us) server_negotiation.cc:407] Connection header received
0809 02:14:52.102281 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.102283 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.102320 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.102413 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:52.102876 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.103569 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105046 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.105758 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105851 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.105906 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.106013 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.106054 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":102,"threads_started":1}
I20260809 02:14:52.106294 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.102149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44160 (local address 127.9.40.67:40533)
0809 02:14:52.102365 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.102369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.102410 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:14:52.102503 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.102505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.102523 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.102598 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:52.102848 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.103537 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.104829 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.105551 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105635 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.105699 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.106165 (+ 466us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.106211 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":89,"threads_started":1}
I20260809 02:14:52.106375 14497 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.101700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43421 (local address 127.0.0.1:37856)
0809 02:14:52.102142 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.102151 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.102209 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.102532 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.102536 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.102538 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:52.102630 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.102632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.103694 (+ 1062us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.103695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.104959 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.104961 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.105853 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105855 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.105931 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.105936 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.105997 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.106039 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"mutex_wait_us":32,"thread_start_us":94,"threads_started":1}
I20260809 02:14:52.107095 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.101698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44183 (local address 127.0.0.1:38360)
0809 02:14:52.101866 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.101909 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.101976 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.102698 (+ 722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.102699 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.102700 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:52.102780 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.102783 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.104948 (+ 2165us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.104949 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.105701 (+ 752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.105703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.106864 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.106865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.106956 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.106960 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.106991 (+ 31us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.107028 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"mutex_wait_us":5,"thread_start_us":62,"threads_started":1}
I20260809 02:14:52.107234 14116 tablet_service.cc:1467] Processing CreateTablet for tablet 0f39b8f863a246239e4f3067dd889b4e (DEFAULT_TABLE table=test-table [id=2e30168292ec44cfaa885048adff6f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.107226 14412 tablet_service.cc:1467] Processing CreateTablet for tablet 0f39b8f863a246239e4f3067dd889b4e (DEFAULT_TABLE table=test-table [id=2e30168292ec44cfaa885048adff6f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.107447 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.102172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38360 (local address 127.9.40.66:44183)
0809 02:14:52.102558 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.102562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.102572 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:52.102611 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.102613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.102627 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.102705 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:52.103636 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.104820 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.105807 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.106764 (+ 957us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.107222 (+ 458us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.107288 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.107316 (+ 28us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.107355 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":95,"threads_started":1}
I20260809 02:14:52.107654 14116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f39b8f863a246239e4f3067dd889b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.107656 14412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f39b8f863a246239e4f3067dd889b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.110664 14500 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: Bootstrap starting.
I20260809 02:14:52.110901 14501 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Bootstrap starting.
I20260809 02:14:52.111387 14265 tablet_service.cc:1467] Processing CreateTablet for tablet 0f39b8f863a246239e4f3067dd889b4e (DEFAULT_TABLE table=test-table [id=2e30168292ec44cfaa885048adff6f70]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.111514 14500 tablet_bootstrap.cc:654] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.111575 14501 tablet_bootstrap.cc:654] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.111758 14265 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f39b8f863a246239e4f3067dd889b4e. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.112237 14500 log.cc:824] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.112283 14501 log.cc:824] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.113081 14500 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: No bootstrap required, opened a new log
I20260809 02:14:52.113335 14500 ts_tablet_manager.cc:1397] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:14:52.113353 14501 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: No bootstrap required, opened a new log
I20260809 02:14:52.113411 14501 ts_tablet_manager.cc:1397] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:14:52.114902 14504 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Bootstrap starting.
I20260809 02:14:52.115491 14504 tablet_bootstrap.cc:654] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.115450 14500 raft_consensus.cc:348] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.115450 14501 raft_consensus.cc:348] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.115628 14500 raft_consensus.cc:374] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.115628 14501 raft_consensus.cc:374] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.115680 14501 raft_consensus.cc:729] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 63e7824a818b40d9acf2f413cc6f2787, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.115710 14500 raft_consensus.cc:729] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1d2ff67c23494901bb94010a01561bde, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.115835 14500 consensus_queue.cc:260] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.115899 14501 consensus_queue.cc:260] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.116137 14504 log.cc:824] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.116164 14500 ts_tablet_manager.cc:1428] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:52.116158 14501 ts_tablet_manager.cc:1428] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:52.116283 14483 heartbeater.cc:503] Master 127.9.40.126:44949 was elected leader, sending a full tablet report...
I20260809 02:14:52.116312 14189 heartbeater.cc:503] Master 127.9.40.126:44949 was elected leader, sending a full tablet report...
I20260809 02:14:52.116997 14504 tablet_bootstrap.cc:492] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: No bootstrap required, opened a new log
I20260809 02:14:52.117133 14504 ts_tablet_manager.cc:1397] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:52.118794 14504 raft_consensus.cc:348] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.118923 14504 raft_consensus.cc:374] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.118947 14504 raft_consensus.cc:729] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f589751780b7480c849cf0f96b427b68, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.119092 14504 consensus_queue.cc:260] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.119366 14504 ts_tablet_manager.cc:1428] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:52.119385 14338 heartbeater.cc:503] Master 127.9.40.126:44949 was elected leader, sending a full tablet report...
I20260809 02:14:52.120944 14505 raft_consensus.cc:482] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:52.121018 14505 raft_consensus.cc:504] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.121290 14505 leader_election.cc:288] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f589751780b7480c849cf0f96b427b68 (127.9.40.66:44183), 63e7824a818b40d9acf2f413cc6f2787 (127.9.40.67:40533)
I20260809 02:14:52.124595 14193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.121427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40533 (local address 127.9.40.65:39545)
0809 02:14:52.121468 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.121483 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.121526 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.121773 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.121774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.121776 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:52.121852 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.121854 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.122567 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.122568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.123728 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.123731 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.124299 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.124300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.124384 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.124392 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.124436 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.124489 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 02:14:52.124614 14498 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.121524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39545 (local address 127.9.40.67:40533)
0809 02:14:52.121595 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.121598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.121608 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:52.121634 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.121635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.121648 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.121714 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:52.121923 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.122493 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.123810 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.124211 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.124297 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.124307 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.124461 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.124506 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:14:52.124639 14499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.121608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36109 (local address 127.9.40.66:44183)
0809 02:14:52.121676 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.121678 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.121752 (+ 74us) server_negotiation.cc:407] Connection header received
0809 02:14:52.121799 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.121800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.121811 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.121877 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:52.122156 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.122633 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.123859 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.124210 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.124297 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.124307 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.124499 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.124558 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:14:52.124624 14509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.121447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44183 (local address 127.9.40.65:36109)
0809 02:14:52.121682 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.121692 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.121749 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.122003 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.122004 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.122007 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:52.122075 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.122077 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.122738 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.122739 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.123745 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:52.123755 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.124300 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.124301 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:52.124401 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.124405 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.124482 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.124519 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260809 02:14:52.125022 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f39b8f863a246239e4f3067dd889b4e" candidate_uuid: "1d2ff67c23494901bb94010a01561bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f589751780b7480c849cf0f96b427b68" is_pre_election: true
I20260809 02:14:52.125070 14434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f39b8f863a246239e4f3067dd889b4e" candidate_uuid: "1d2ff67c23494901bb94010a01561bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e7824a818b40d9acf2f413cc6f2787" is_pre_election: true
I20260809 02:14:52.125144 14288 raft_consensus.cc:2454] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d2ff67c23494901bb94010a01561bde in term 0.
I20260809 02:14:52.125218 14434 raft_consensus.cc:2454] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1d2ff67c23494901bb94010a01561bde in term 0.
I20260809 02:14:52.125350 14065 leader_election.cc:302] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2ff67c23494901bb94010a01561bde, f589751780b7480c849cf0f96b427b68; no voters:
I20260809 02:14:52.125480 14505 raft_consensus.cc:2781] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:52.125536 14505 raft_consensus.cc:482] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:52.125559 14505 raft_consensus.cc:3037] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.126246 14505 raft_consensus.cc:504] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.126396 14505 leader_election.cc:288] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [CANDIDATE]: Term 1 election: Requested vote from peers f589751780b7480c849cf0f96b427b68 (127.9.40.66:44183), 63e7824a818b40d9acf2f413cc6f2787 (127.9.40.67:40533)
I20260809 02:14:52.126559 14288 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f39b8f863a246239e4f3067dd889b4e" candidate_uuid: "1d2ff67c23494901bb94010a01561bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f589751780b7480c849cf0f96b427b68"
I20260809 02:14:52.126622 14288 raft_consensus.cc:3037] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.127329 14288 raft_consensus.cc:2454] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d2ff67c23494901bb94010a01561bde in term 1.
I20260809 02:14:52.127497 14065 leader_election.cc:302] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1d2ff67c23494901bb94010a01561bde, f589751780b7480c849cf0f96b427b68; no voters:
I20260809 02:14:52.127481 14434 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f39b8f863a246239e4f3067dd889b4e" candidate_uuid: "1d2ff67c23494901bb94010a01561bde" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "63e7824a818b40d9acf2f413cc6f2787"
I20260809 02:14:52.127586 14505 raft_consensus.cc:2781] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:52.127558 14434 raft_consensus.cc:3037] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.127784 14505 raft_consensus.cc:686] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 1 LEADER]: Becoming Leader. State: Replica: 1d2ff67c23494901bb94010a01561bde, State: Running, Role: LEADER
I20260809 02:14:52.127872 14505 consensus_queue.cc:237] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } }
I20260809 02:14:52.128499 14434 raft_consensus.cc:2454] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1d2ff67c23494901bb94010a01561bde in term 1.
I20260809 02:14:52.128682 13971 catalog_manager.cc:5495] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde reported cstate change: term changed from 0 to 1, leader changed from <none> to 1d2ff67c23494901bb94010a01561bde (127.9.40.65). New cstate: current_term: 1 leader_uuid: "1d2ff67c23494901bb94010a01561bde" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1d2ff67c23494901bb94010a01561bde" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43421 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 } health_report { overall_health: UNKNOWN } } }
I20260809 02:14:52.141373 14496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.139076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37860 (local address 127.9.40.65:43421)
0809 02:14:52.139146 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.139148 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.139181 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:14:52.139259 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.139260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.139278 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.139342 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:14:52.139471 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.139929 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.140629 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:52.140878 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:52.140932 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:52.141079 (+ 147us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:14:52.141184 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:14:52.141244 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.141282 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.141299 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:14:52.143234 14434 raft_consensus.cc:1261] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 1 FOLLOWER]: Refusing update from remote peer 1d2ff67c23494901bb94010a01561bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:52.143249 14288 raft_consensus.cc:1261] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 1 FOLLOWER]: Refusing update from remote peer 1d2ff67c23494901bb94010a01561bde: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:52.143627 14510 consensus_queue.cc:1035] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "63e7824a818b40d9acf2f413cc6f2787" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40533 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:52.143754 14505 consensus_queue.cc:1035] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [LEADER]: Connected to new peer: Peer: permanent_uuid: "f589751780b7480c849cf0f96b427b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44183 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:52.146178 14514 mvcc.cc:205] Tried to move back new op lower bound from 7316445971015622656 to 7316445970956296192. Current Snapshot: MvccSnapshot[applied={T|T < 7316445971015622656}]
I20260809 02:14:52.147219 14515 mvcc.cc:205] Tried to move back new op lower bound from 7316445971015622656 to 7316445970956296192. Current Snapshot: MvccSnapshot[applied={T|T < 7316445971015622656 or (T in {7316445971015622656})}]
I20260809 02:14:52.147486 14516 mvcc.cc:205] Tried to move back new op lower bound from 7316445971015622656 to 7316445970956296192. Current Snapshot: MvccSnapshot[applied={T|T < 7316445971015622656 or (T in {7316445971015622656})}]
I20260809 02:14:52.148634 13969 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:52.148785 13969 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48696:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:52.149550 13969 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f9f9af904dca4f00ac218b68ac0d9b56: Sending DeleteTablet for 3 replicas of tablet 0f39b8f863a246239e4f3067dd889b4e
I20260809 02:14:52.149906 14265 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f39b8f863a246239e4f3067dd889b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38360
I20260809 02:14:52.150023 14116 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f39b8f863a246239e4f3067dd889b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37856
I20260809 02:14:52.150213 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14044
I20260809 02:14:52.150270 14525 tablet_replica.cc:330] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde: stopping tablet replica
I20260809 02:14:52.150264 14412 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f39b8f863a246239e4f3067dd889b4e with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:44160
I20260809 02:14:52.150364 14525 raft_consensus.cc:2229] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:52.150463 14526 tablet_replica.cc:330] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: stopping tablet replica
I20260809 02:14:52.150465 14525 raft_consensus.cc:2258] T 0f39b8f863a246239e4f3067dd889b4e P 1d2ff67c23494901bb94010a01561bde [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.150542 14526 raft_consensus.cc:2229] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:52.150645 14526 raft_consensus.cc:2258] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.150879 14523 tablet_replica.cc:330] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: stopping tablet replica
I20260809 02:14:52.150961 14523 raft_consensus.cc:2229] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:52.151027 14523 raft_consensus.cc:2258] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.151757 14526 ts_tablet_manager.cc:1905] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:52.151779 14523 ts_tablet_manager.cc:1905] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:52.153329 14523 ts_tablet_manager.cc:1918] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:52.153384 14523 log.cc:1196] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/0f39b8f863a246239e4f3067dd889b4e
I20260809 02:14:52.153682 14523 ts_tablet_manager.cc:1939] T 0f39b8f863a246239e4f3067dd889b4e P f589751780b7480c849cf0f96b427b68: Deleting consensus metadata
I20260809 02:14:52.153947 14526 ts_tablet_manager.cc:1918] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:52.154016 14526 log.cc:1196] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/0f39b8f863a246239e4f3067dd889b4e
I20260809 02:14:52.154150 13955 catalog_manager.cc:4841] TS f589751780b7480c849cf0f96b427b68 (127.9.40.66:44183): tablet 0f39b8f863a246239e4f3067dd889b4e (table test-table [id=2e30168292ec44cfaa885048adff6f70]) successfully deleted
I20260809 02:14:52.154361 14526 ts_tablet_manager.cc:1939] T 0f39b8f863a246239e4f3067dd889b4e P 63e7824a818b40d9acf2f413cc6f2787: Deleting consensus metadata
I20260809 02:14:52.154776 13957 catalog_manager.cc:4841] TS 63e7824a818b40d9acf2f413cc6f2787 (127.9.40.67:40533): tablet 0f39b8f863a246239e4f3067dd889b4e (table test-table [id=2e30168292ec44cfaa885048adff6f70]) successfully deleted
W20260809 02:14:52.159531 13957 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:43421 (error 108)
I20260809 02:14:52.159775 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14196
I20260809 02:14:52.161010 14495 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.159961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43421 (local address 127.0.0.1:37862)
0809 02:14:52.160913 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.160964 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43421: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":929}
W20260809 02:14:52.161244 13957 catalog_manager.cc:4584] TS 1d2ff67c23494901bb94010a01561bde (127.9.40.65:43421): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f39b8f863a246239e4f3067dd889b4e: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43421: connect: Connection refused (error 111)
I20260809 02:14:52.165611 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14346
I20260809 02:14:52.172432 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 13939
2026-08-09T02:14:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (1984 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-09T02:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:52Z Disabled control of system clock
I20260809 02:14:52.203255 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40317
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:14:52.286805 14535 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:52.287042 14535 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:52.287065 14535 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:52.288313 14535 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:52.288365 14535 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:52.288378 14535 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:52.288391 14535 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:52.288447 14535 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:52.289880 14535 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40317
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14535
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:52.290172 14535 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:52.290506 14535 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:52.295564 14544 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.295549 14541 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.295550 14540 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.295971 14535 server_base.cc:1034] running on GCE node
I20260809 02:14:52.296301 14535 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:52.296557 14535 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:52.297695 14535 hybrid_clock.cc:648] HybridClock initialized: now 1786241692297672 us; error 45 us; skew 500 ppm
I20260809 02:14:52.298866 14535 webserver.cc:460] Webserver started at http://127.9.40.126:43541/ using document root <none> and password file <none>
I20260809 02:14:52.299077 14535 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:52.299137 14535 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:52.299233 14535 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:52.300274 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "1844e3330be44b45a1c1197f1ebe05f4"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "040775c9c0beace011ce3837dba65ed8"
server_key_iv: "1e7a4eaa0efb2e06b1e3685dc3b7deeb"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.300585 14535 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "1844e3330be44b45a1c1197f1ebe05f4"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "040775c9c0beace011ce3837dba65ed8"
server_key_iv: "1e7a4eaa0efb2e06b1e3685dc3b7deeb"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.301913 14535 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:52.302583 14551 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.302758 14535 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:52.302838 14535 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "1844e3330be44b45a1c1197f1ebe05f4"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "040775c9c0beace011ce3837dba65ed8"
server_key_iv: "1e7a4eaa0efb2e06b1e3685dc3b7deeb"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.302915 14535 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:52.306267 14535 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:52.306543 14535 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:52.306692 14535 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:52.310732 14535 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:40317
I20260809 02:14:52.310777 14607 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:40317 every 8 connection(s)
I20260809 02:14:52.311111 14535 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:52.311810 14608 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.314570 14608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: Bootstrap starting.
I20260809 02:14:52.315145 14608 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.315430 14608 log.cc:824] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.316157 14608 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: No bootstrap required, opened a new log
I20260809 02:14:52.317927 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14535
I20260809 02:14:52.318014 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:52.318216 9377 external_mini_cluster.cc:1428] Setting key 2e2d5fe3ea9486ca3be4121df18c74f2
I20260809 02:14:52.318177 14608 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } }
I20260809 02:14:52.318358 14608 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.318470 14608 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1844e3330be44b45a1c1197f1ebe05f4, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.318650 14608 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } }
I20260809 02:14:52.318761 14608 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:52.318837 14608 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:52.318900 14608 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.319576 14608 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } }
I20260809 02:14:52.319703 14608 leader_election.cc:302] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1844e3330be44b45a1c1197f1ebe05f4; no voters:
I20260809 02:14:52.319890 14608 leader_election.cc:288] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:52.320005 14614 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:52.320191 14614 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [term 1 LEADER]: Becoming Leader. State: Replica: 1844e3330be44b45a1c1197f1ebe05f4, State: Running, Role: LEADER
I20260809 02:14:52.320199 14608 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:52.320339 14614 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } }
I20260809 02:14:52.320850 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.318772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42718 (local address 127.9.40.126:40317)
0809 02:14:52.319025 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.319034 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.319071 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:14:52.319174 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.319179 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.319211 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.319331 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.320143 (+ 812us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.320147 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.320147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.320152 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.320195 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.320210 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.320272 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.320537 (+ 265us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.320599 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":104,"threads_started":1}
I20260809 02:14:52.321112 14616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1844e3330be44b45a1c1197f1ebe05f4. Latest consensus state: current_term: 1 leader_uuid: "1844e3330be44b45a1c1197f1ebe05f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } } }
I20260809 02:14:52.321210 14616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:52.321542 14615 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1844e3330be44b45a1c1197f1ebe05f4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1844e3330be44b45a1c1197f1ebe05f4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40317 } } }
I20260809 02:14:52.321612 14615 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4 [sys.catalog]: This master's current role is: LEADER
W20260809 02:14:52.322320 14630 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:52.322366 14630 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:52.322440 14624 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:52.322588 14624 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:52.324045 14624 catalog_manager.cc:1347] Generated new cluster ID: 30369a3924e243d18d84e57b5218bdfc
I20260809 02:14:52.324101 14624 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:52.331174 14624 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:52.331377 14624 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:52.335619 14624 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: Generated new TSK 0
I20260809 02:14:52.335821 14624 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:52.337807 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--builtin_ntp_servers=127.9.40.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:14:52.421145 14634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:52.421394 14634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:52.421418 14634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:52.422591 14634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:52.422633 14634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:52.422659 14634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:52.424011 14634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:52.424250 14634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:52.424566 14634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:52.429883 14639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.429956 14640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.430040 14642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.430310 14634 server_base.cc:1034] running on GCE node
I20260809 02:14:52.430510 14634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:52.430783 14634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:52.431957 14634 hybrid_clock.cc:648] HybridClock initialized: now 1786241692431933 us; error 35 us; skew 500 ppm
I20260809 02:14:52.433473 14634 webserver.cc:460] Webserver started at http://127.9.40.65:35891/ using document root <none> and password file <none>
I20260809 02:14:52.433738 14634 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:52.433785 14634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:52.433951 14634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:52.435075 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "1e824d9387584a3da103dd201e128b3b"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "a59ce7b3ed79c893bea189db5fa298f3"
server_key_iv: "27c42bd3cb218e81e54e2707a1d978f3"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.435456 14634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "1e824d9387584a3da103dd201e128b3b"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "a59ce7b3ed79c893bea189db5fa298f3"
server_key_iv: "27c42bd3cb218e81e54e2707a1d978f3"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.436869 14634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 02:14:52.437718 14649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.437878 14634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:52.437986 14634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "1e824d9387584a3da103dd201e128b3b"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "a59ce7b3ed79c893bea189db5fa298f3"
server_key_iv: "27c42bd3cb218e81e54e2707a1d978f3"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.438069 14634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:52.441541 14634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:52.441850 14634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:52.441989 14634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:52.442242 14634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:52.442633 14634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:52.442672 14634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.442693 14634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:52.442735 14634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.450009 14634 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:35049
I20260809 02:14:52.450053 14770 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:35049 every 8 connection(s)
I20260809 02:14:52.450440 14634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:52.453516 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14634
I20260809 02:14:52.453647 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:52.453855 9377 external_mini_cluster.cc:1428] Setting key 8fb6cd99c753e2b9948ba3f17588b2d9
I20260809 02:14:52.454715 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.451238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:41739 (local address 127.9.40.126:40317)
0809 02:14:52.451326 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.451329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.452427 (+ 1098us) server_negotiation.cc:407] Connection header received
0809 02:14:52.452987 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.452988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.452995 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.453052 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.454437 (+ 1385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.454438 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.454438 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.454443 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.454462 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.454466 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.454527 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.454630 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.454647 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260809 02:14:52.454965 14773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.451247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40317 (local address 127.9.40.65:41739)
0809 02:14:52.452289 (+ 1042us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.452327 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.452896 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.453108 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.453111 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.453119 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.454224 (+ 1105us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.454225 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.454255 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.454313 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.454546 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.454553 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.454556 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.454650 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.454723 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":976,"thread_start_us":99,"threads_started":1}
I20260809 02:14:52.455050 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--builtin_ntp_servers=127.9.40.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260809 02:14:52.455559 14771 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40317
I20260809 02:14:52.455677 14771 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:52.455852 14771 heartbeater.cc:511] Master 127.9.40.126:40317 requested a full tablet report, sending...
I20260809 02:14:52.456456 14569 ts_manager.cc:194] Registered new tserver with Master: 1e824d9387584a3da103dd201e128b3b (127.9.40.65:35049)
W20260809 02:14:52.538287 14774 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:52.538496 14774 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:52.538522 14774 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:52.539611 14774 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:52.539646 14774 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:52.539671 14774 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:52.541100 14774 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14774
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:52.541296 14774 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:52.541533 14774 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:52.546710 14782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.546710 14779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.546873 14774 server_base.cc:1034] running on GCE node
W20260809 02:14:52.546919 14780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.547206 14774 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:52.547462 14774 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:52.548629 14774 hybrid_clock.cc:648] HybridClock initialized: now 1786241692548618 us; error 45 us; skew 500 ppm
I20260809 02:14:52.549988 14774 webserver.cc:460] Webserver started at http://127.9.40.66:44333/ using document root <none> and password file <none>
I20260809 02:14:52.550262 14774 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:52.550334 14774 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:52.550493 14774 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:52.551556 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "712072b1553e47258df629a2dd76782d"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "fd042df707c9eff2c038fc0694c06483"
server_key_iv: "94cc8fd301ea291088b6d8d14e7f641d"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.551923 14774 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "712072b1553e47258df629a2dd76782d"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "fd042df707c9eff2c038fc0694c06483"
server_key_iv: "94cc8fd301ea291088b6d8d14e7f641d"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.553333 14774 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:52.554173 14788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.554453 14774 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:52.554557 14774 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "712072b1553e47258df629a2dd76782d"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "fd042df707c9eff2c038fc0694c06483"
server_key_iv: "94cc8fd301ea291088b6d8d14e7f641d"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.554637 14774 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:52.558046 14774 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:52.558312 14774 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:52.558454 14774 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:52.558660 14774 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:52.558967 14774 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:52.559020 14774 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.559062 14774 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:52.559096 14774 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.565120 14774 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:43263
I20260809 02:14:52.565164 14913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:43263 every 8 connection(s)
I20260809 02:14:52.565516 14774 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:52.568799 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.566245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:33815 (local address 127.9.40.126:40317)
0809 02:14:52.566292 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.566294 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.566556 (+ 262us) server_negotiation.cc:407] Connection header received
0809 02:14:52.567145 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.567146 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.567154 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.567215 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.568487 (+ 1272us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.568489 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.568489 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.568494 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.568513 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.568517 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.568578 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.568685 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.568703 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:14:52.569010 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.566207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40317 (local address 127.9.40.66:33815)
0809 02:14:52.566405 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.566456 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.566999 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.567287 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.567291 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.567301 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.568323 (+ 1022us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.568324 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.568354 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.568381 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.568585 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.568598 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.568601 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.568697 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.568780 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":71,"threads_started":1}
I20260809 02:14:52.569495 14914 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40317
I20260809 02:14:52.569617 14914 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:52.569770 14914 heartbeater.cc:511] Master 127.9.40.126:40317 requested a full tablet report, sending...
I20260809 02:14:52.570142 14569 ts_manager.cc:194] Registered new tserver with Master: 712072b1553e47258df629a2dd76782d (127.9.40.66:43263)
I20260809 02:14:52.570621 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14774
I20260809 02:14:52.570703 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:52.570844 9377 external_mini_cluster.cc:1428] Setting key d72e07dd2de3c5d8ea12d62cbeea4ea9
I20260809 02:14:52.571646 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--builtin_ntp_servers=127.9.40.84:43831
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260809 02:14:52.656237 14917 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:52.656455 14917 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:52.656479 14917 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:52.657840 14917 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:52.657886 14917 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:52.657913 14917 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:52.659421 14917 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43831
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14917
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:52.659636 14917 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:52.659909 14917 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:52.665186 14924 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.665233 14922 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:52.665175 14926 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:52.665339 14917 server_base.cc:1034] running on GCE node
I20260809 02:14:52.665555 14917 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:52.665762 14917 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:52.666937 14917 hybrid_clock.cc:648] HybridClock initialized: now 1786241692666914 us; error 37 us; skew 500 ppm
I20260809 02:14:52.668272 14917 webserver.cc:460] Webserver started at http://127.9.40.67:37327/ using document root <none> and password file <none>
I20260809 02:14:52.668530 14917 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:52.668623 14917 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:52.668803 14917 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:52.669899 14917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "0276275697d24aca80ec6bbbd67a66ce"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "c22b8aa7c0a22397c64aa9c820ca598c"
server_key_iv: "6926c9fbb0d0cfff5c014958937db3f8"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.670260 14917 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "0276275697d24aca80ec6bbbd67a66ce"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "c22b8aa7c0a22397c64aa9c820ca598c"
server_key_iv: "6926c9fbb0d0cfff5c014958937db3f8"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.671657 14917 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:52.672386 14933 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.672565 14917 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:52.672678 14917 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "0276275697d24aca80ec6bbbd67a66ce"
format_stamp: "Formatted at 2026-08-09 02:14:52 on dist-test-slave-qg60"
server_key: "c22b8aa7c0a22397c64aa9c820ca598c"
server_key_iv: "6926c9fbb0d0cfff5c014958937db3f8"
server_key_version: "encryptionkey@0"
I20260809 02:14:52.672785 14917 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:52.676211 14917 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:52.676506 14917 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:52.676666 14917 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:52.676915 14917 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:52.677249 14917 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:52.677306 14917 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.677346 14917 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:52.677376 14917 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:52.683305 14917 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:34409
I20260809 02:14:52.683357 15061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:34409 every 8 connection(s)
I20260809 02:14:52.683727 14917 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:52.686851 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.684491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48567 (local address 127.9.40.126:40317)
0809 02:14:52.684536 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.684538 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.684793 (+ 255us) server_negotiation.cc:407] Connection header received
0809 02:14:52.685330 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.685332 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.685339 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.685400 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.686571 (+ 1171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.686572 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.686573 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.686577 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.686591 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.686595 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.686650 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.686761 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.686778 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:52.687043 15064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.684458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40317 (local address 127.9.40.67:48567)
0809 02:14:52.684640 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.684675 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.685216 (+ 541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.685476 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.685481 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.685493 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.686411 (+ 918us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.686412 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.686441 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.686468 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.686655 (+ 187us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.686660 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.686663 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.686755 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.686819 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":77,"threads_started":1}
I20260809 02:14:52.687130 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 14917
I20260809 02:14:52.687222 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:52.687489 9377 external_mini_cluster.cc:1428] Setting key e801a08dea8809bdec6083e20ae073a6
I20260809 02:14:52.687546 15062 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40317
I20260809 02:14:52.687641 15062 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:52.687798 15062 heartbeater.cc:511] Master 127.9.40.126:40317 requested a full tablet report, sending...
I20260809 02:14:52.688226 14568 ts_manager.cc:194] Registered new tserver with Master: 0276275697d24aca80ec6bbbd67a66ce (127.9.40.67:34409)
I20260809 02:14:52.699189 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:14:52.701779 14613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.700664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42732 (local address 127.9.40.126:40317)
0809 02:14:52.700713 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.700715 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.700845 (+ 130us) server_negotiation.cc:407] Connection header received
0809 02:14:52.700930 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.700931 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.700936 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.700986 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.701623 (+ 637us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.701624 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.701624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.701627 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.701639 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.701642 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.701681 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.701716 (+ 35us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.701728 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:14:52.703436 14568 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42732:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:14:52.705111 14568 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:14:52.709420 15071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43263 (local address 127.0.0.1:56558)
0809 02:14:52.707643 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.707656 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.707704 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.707952 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.707960 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.707967 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.708502 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.708503 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.708981 (+ 478us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.708994 (+ 13us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.709276 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.709277 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.709278 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.709331 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.709341 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":125,"threads_started":1}
I20260809 02:14:52.709518 15073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41322 (local address 127.9.40.65:35049)
0809 02:14:52.707872 (+ 422us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.708185 (+ 313us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.708199 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:14:52.708227 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.708230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.708240 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.708337 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.709119 (+ 782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.709125 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.709125 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.709130 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.709160 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.709187 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.709381 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.709402 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.709417 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":379,"thread_start_us":92,"threads_started":1}
I20260809 02:14:52.709580 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37096 (local address 127.9.40.67:34409)
0809 02:14:52.707952 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.707956 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.707966 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:52.707989 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.707991 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.708000 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.708072 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.709055 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.709066 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.709067 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.709072 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.709110 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.709163 (+ 53us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.709235 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.709462 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.709479 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":93,"threads_started":1}
I20260809 02:14:52.709420 15072 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:35049 (local address 127.0.0.1:41322)
0809 02:14:52.707551 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.707594 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.707672 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.708354 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.708355 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.708358 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.709000 (+ 642us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.709000 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.709024 (+ 24us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.709040 (+ 16us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.709270 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.709271 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.709272 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.709305 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.709320 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":71,"threads_started":1}
I20260809 02:14:52.709715 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34409 (local address 127.0.0.1:37096)
0809 02:14:52.707784 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.707794 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.707827 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.708363 (+ 536us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.708364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.708367 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.708900 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.708901 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.708931 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.708961 (+ 30us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.709241 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.709248 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.709250 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.709317 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.709331 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":76,"threads_started":1}
I20260809 02:14:52.709446 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.707641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56558 (local address 127.9.40.66:43263)
0809 02:14:52.707809 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.707814 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.707826 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:52.707854 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.707856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.707865 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.707941 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.709097 (+ 1156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.709109 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.709109 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.709114 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.709154 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.709184 (+ 30us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.709281 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.709340 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.709356 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":99,"threads_started":1}
I20260809 02:14:52.710423 14987 tablet_service.cc:1467] Processing CreateTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 (DEFAULT_TABLE table=test-table [id=d155fa5e4c294f4dbd15acf96e18a962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.710578 14701 tablet_service.cc:1467] Processing CreateTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 (DEFAULT_TABLE table=test-table [id=d155fa5e4c294f4dbd15acf96e18a962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.710834 14987 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e97b8a2b7b4081833a9bd930c685e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.710739 14842 tablet_service.cc:1467] Processing CreateTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 (DEFAULT_TABLE table=test-table [id=d155fa5e4c294f4dbd15acf96e18a962]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:14:52.710965 14701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e97b8a2b7b4081833a9bd930c685e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.711073 14842 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 85e97b8a2b7b4081833a9bd930c685e1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:52.713858 15078 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: Bootstrap starting.
I20260809 02:14:52.714320 15079 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Bootstrap starting.
I20260809 02:14:52.714605 15080 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Bootstrap starting.
I20260809 02:14:52.714818 15078 tablet_bootstrap.cc:654] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.714955 15079 tablet_bootstrap.cc:654] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.715302 15080 tablet_bootstrap.cc:654] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:52.715322 15079 log.cc:824] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.715375 15078 log.cc:824] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.715631 15080 log.cc:824] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:52.716149 15078 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: No bootstrap required, opened a new log
I20260809 02:14:52.716205 15078 ts_tablet_manager.cc:1397] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:14:52.716413 15080 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: No bootstrap required, opened a new log
I20260809 02:14:52.716495 15080 ts_tablet_manager.cc:1397] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:14:52.717116 15079 tablet_bootstrap.cc:492] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: No bootstrap required, opened a new log
I20260809 02:14:52.717221 15079 ts_tablet_manager.cc:1397] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260809 02:14:52.718501 15080 raft_consensus.cc:348] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.718689 15080 raft_consensus.cc:374] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.718741 15080 raft_consensus.cc:729] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 712072b1553e47258df629a2dd76782d, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.718874 15080 consensus_queue.cc:260] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.718864 15078 raft_consensus.cc:348] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.719025 15078 raft_consensus.cc:374] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.719063 15078 raft_consensus.cc:729] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1e824d9387584a3da103dd201e128b3b, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.719172 15080 ts_tablet_manager.cc:1428] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:52.719185 15078 consensus_queue.cc:260] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.719302 14914 heartbeater.cc:503] Master 127.9.40.126:40317 was elected leader, sending a full tablet report...
I20260809 02:14:52.719465 14771 heartbeater.cc:503] Master 127.9.40.126:40317 was elected leader, sending a full tablet report...
I20260809 02:14:52.719447 15078 ts_tablet_manager.cc:1428] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:14:52.720310 15079 raft_consensus.cc:348] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.720487 15079 raft_consensus.cc:374] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:52.720530 15079 raft_consensus.cc:729] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0276275697d24aca80ec6bbbd67a66ce, State: Initialized, Role: FOLLOWER
I20260809 02:14:52.720654 15079 consensus_queue.cc:260] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.720963 15079 ts_tablet_manager.cc:1428] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 02:14:52.721015 15062 heartbeater.cc:503] Master 127.9.40.126:40317 was elected leader, sending a full tablet report...
W20260809 02:14:52.816236 14915 tablet.cc:2369] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:14:52.873342 15085 raft_consensus.cc:482] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:14:52.873481 15085 raft_consensus.cc:504] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.873997 15085 leader_election.cc:288] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1e824d9387584a3da103dd201e128b3b (127.9.40.65:35049), 0276275697d24aca80ec6bbbd67a66ce (127.9.40.67:34409)
I20260809 02:14:52.875504 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.874250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:35049 (local address 127.9.40.66:55453)
0809 02:14:52.874340 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.874356 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.874423 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.874593 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.874594 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.874597 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.875071 (+ 474us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.875072 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.875080 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.875107 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.875347 (+ 240us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.875348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.875348 (+ 0us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.875410 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.875425 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260809 02:14:52.875545 15073 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.874381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:55453 (local address 127.9.40.65:35049)
0809 02:14:52.874454 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.874456 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.874466 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:52.874494 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.874496 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.874501 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.874566 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.875218 (+ 652us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.875220 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.875220 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.875224 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.875248 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.875253 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.875338 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.875442 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.875458 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:14:52.875542 15076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.874465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:51613 (local address 127.9.40.67:34409)
0809 02:14:52.874528 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.874530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.874667 (+ 137us) server_negotiation.cc:407] Connection header received
0809 02:14:52.874734 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.874735 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.874758 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.874820 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.875312 (+ 492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.875313 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.875313 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.875318 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.875332 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.875336 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.875416 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.875463 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.875475 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:14:52.875699 15088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.874381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:34409 (local address 127.9.40.66:51613)
0809 02:14:52.874580 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.874590 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:52.874671 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:52.874830 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:52.874831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:52.874833 (+ 2us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:52.875215 (+ 382us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:14:52.875216 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:52.875225 (+ 9us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:14:52.875233 (+ 8us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:52.875415 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:52.875415 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:52.875416 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:14:52.875461 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 02:14:52.875472 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":106,"threads_started":1}
I20260809 02:14:52.875849 14724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e97b8a2b7b4081833a9bd930c685e1" candidate_uuid: "712072b1553e47258df629a2dd76782d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e824d9387584a3da103dd201e128b3b" is_pre_election: true
I20260809 02:14:52.875999 14724 raft_consensus.cc:2454] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712072b1553e47258df629a2dd76782d in term 0.
I20260809 02:14:52.876180 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e97b8a2b7b4081833a9bd930c685e1" candidate_uuid: "712072b1553e47258df629a2dd76782d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0276275697d24aca80ec6bbbd67a66ce" is_pre_election: true
I20260809 02:14:52.876271 14790 leader_election.cc:302] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1e824d9387584a3da103dd201e128b3b, 712072b1553e47258df629a2dd76782d; no voters:
I20260809 02:14:52.876349 15011 raft_consensus.cc:2454] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 712072b1553e47258df629a2dd76782d in term 0.
I20260809 02:14:52.876410 15085 raft_consensus.cc:2781] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:14:52.876477 15085 raft_consensus.cc:482] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:14:52.876499 15085 raft_consensus.cc:3037] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.877326 15085 raft_consensus.cc:504] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.877446 15085 leader_election.cc:288] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [CANDIDATE]: Term 1 election: Requested vote from peers 1e824d9387584a3da103dd201e128b3b (127.9.40.65:35049), 0276275697d24aca80ec6bbbd67a66ce (127.9.40.67:34409)
I20260809 02:14:52.877624 14724 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e97b8a2b7b4081833a9bd930c685e1" candidate_uuid: "712072b1553e47258df629a2dd76782d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1e824d9387584a3da103dd201e128b3b"
I20260809 02:14:52.877647 15011 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "85e97b8a2b7b4081833a9bd930c685e1" candidate_uuid: "712072b1553e47258df629a2dd76782d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0276275697d24aca80ec6bbbd67a66ce"
I20260809 02:14:52.877710 15011 raft_consensus.cc:3037] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.877712 14724 raft_consensus.cc:3037] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:52.878607 14724 raft_consensus.cc:2454] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712072b1553e47258df629a2dd76782d in term 1.
I20260809 02:14:52.878607 15011 raft_consensus.cc:2454] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 712072b1553e47258df629a2dd76782d in term 1.
I20260809 02:14:52.878801 14790 leader_election.cc:302] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0276275697d24aca80ec6bbbd67a66ce, 712072b1553e47258df629a2dd76782d; no voters:
I20260809 02:14:52.878898 15085 raft_consensus.cc:2781] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:52.879043 15085 raft_consensus.cc:686] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 1 LEADER]: Becoming Leader. State: Replica: 712072b1553e47258df629a2dd76782d, State: Running, Role: LEADER
I20260809 02:14:52.879141 15085 consensus_queue.cc:237] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } }
I20260809 02:14:52.880021 14568 catalog_manager.cc:5495] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d reported cstate change: term changed from 0 to 1, leader changed from <none> to 712072b1553e47258df629a2dd76782d (127.9.40.66). New cstate: current_term: 1 leader_uuid: "712072b1553e47258df629a2dd76782d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "712072b1553e47258df629a2dd76782d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43263 } health_report { overall_health: HEALTHY } } }
I20260809 02:14:52.890803 15075 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.889786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56568 (local address 127.9.40.66:43263)
0809 02:14:52.889853 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:52.889855 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:52.889865 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:52.889913 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:52.889914 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:52.889927 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:52.889992 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:52.890532 (+ 540us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:52.890534 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:52.890534 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:52.890539 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:52.890554 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:52.890558 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:52.890625 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:52.890706 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:14:52.890722 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:14:52.892566 15011 raft_consensus.cc:1261] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 1 FOLLOWER]: Refusing update from remote peer 712072b1553e47258df629a2dd76782d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:52.892619 14724 raft_consensus.cc:1261] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 1 FOLLOWER]: Refusing update from remote peer 712072b1553e47258df629a2dd76782d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:14:52.892983 15089 consensus_queue.cc:1035] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [LEADER]: Connected to new peer: Peer: permanent_uuid: "0276275697d24aca80ec6bbbd67a66ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 34409 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:52.893138 15085 consensus_queue.cc:1035] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [LEADER]: Connected to new peer: Peer: permanent_uuid: "1e824d9387584a3da103dd201e128b3b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 35049 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:14:52.896413 15095 mvcc.cc:205] Tried to move back new op lower bound from 7316445974085361664 to 7316445974033694720. Current Snapshot: MvccSnapshot[applied={T|T < 7316445974085361664 or (T in {7316445974085361664})}]
I20260809 02:14:52.898041 14568 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:52.898146 14568 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42732:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:14:52.898885 14568 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 1844e3330be44b45a1c1197f1ebe05f4: Sending DeleteTablet for 3 replicas of tablet 85e97b8a2b7b4081833a9bd930c685e1
I20260809 02:14:52.899240 14842 tablet_service.cc:1514] Processing DeleteTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='slave'} at 127.0.0.1:56558
I20260809 02:14:52.899482 15104 tablet_replica.cc:330] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: stopping tablet replica
I20260809 02:14:52.899536 14987 tablet_service.cc:1514] Processing DeleteTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='slave'} at 127.0.0.1:37096
I20260809 02:14:52.899613 14701 tablet_service.cc:1514] Processing DeleteTablet for tablet 85e97b8a2b7b4081833a9bd930c685e1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:14:52 UTC) from {username='slave'} at 127.0.0.1:41322
I20260809 02:14:52.899780 15104 raft_consensus.cc:2229] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:14:52.899847 15106 tablet_replica.cc:330] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b: stopping tablet replica
I20260809 02:14:52.899941 15106 raft_consensus.cc:2229] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:52.899977 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14634
I20260809 02:14:52.900041 15106 raft_consensus.cc:2258] T 85e97b8a2b7b4081833a9bd930c685e1 P 1e824d9387584a3da103dd201e128b3b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.900128 15104 raft_consensus.cc:2258] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.905454 15105 tablet_replica.cc:330] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: stopping tablet replica
I20260809 02:14:52.905573 15105 raft_consensus.cc:2229] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:52.905689 15105 raft_consensus.cc:2258] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:52.906035 15104 ts_tablet_manager.cc:1905] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:52.906070 15105 ts_tablet_manager.cc:1905] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:52.908006 15105 ts_tablet_manager.cc:1918] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:52.908093 15105 log.cc:1196] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/85e97b8a2b7b4081833a9bd930c685e1
I20260809 02:14:52.908293 15104 ts_tablet_manager.cc:1918] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:14:52.908389 15105 ts_tablet_manager.cc:1939] T 85e97b8a2b7b4081833a9bd930c685e1 P 0276275697d24aca80ec6bbbd67a66ce: Deleting consensus metadata
I20260809 02:14:52.908478 15104 log.cc:1196] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/85e97b8a2b7b4081833a9bd930c685e1
I20260809 02:14:52.908881 15104 ts_tablet_manager.cc:1939] T 85e97b8a2b7b4081833a9bd930c685e1 P 712072b1553e47258df629a2dd76782d: Deleting consensus metadata
I20260809 02:14:52.909186 14553 catalog_manager.cc:4841] TS 0276275697d24aca80ec6bbbd67a66ce (127.9.40.67:34409): tablet 85e97b8a2b7b4081833a9bd930c685e1 (table test-table [id=d155fa5e4c294f4dbd15acf96e18a962]) successfully deleted
I20260809 02:14:52.909322 14555 catalog_manager.cc:4841] TS 712072b1553e47258df629a2dd76782d (127.9.40.66:43263): tablet 85e97b8a2b7b4081833a9bd930c685e1 (table test-table [id=d155fa5e4c294f4dbd15acf96e18a962]) successfully deleted
W20260809 02:14:52.911139 14553 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:35049 (error 108)
I20260809 02:14:52.911646 15074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:52.911510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:35049 (local address 127.0.0.1:41330)
0809 02:14:52.911565 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:52.911608 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:35049: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:14:52.911715 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14774
W20260809 02:14:52.911849 14553 catalog_manager.cc:4584] TS 1e824d9387584a3da103dd201e128b3b (127.9.40.65:35049): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 85e97b8a2b7b4081833a9bd930c685e1: Network error: Client connection negotiation failed: client connection to 127.9.40.65:35049: connect: Connection refused (error 111)
I20260809 02:14:52.919488 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14917
I20260809 02:14:52.930562 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 14535
2026-08-09T02:14:52Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (757 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260809 02:14:52.953275 9377 mini_oidc.cc:152] Starting JWKS server
I20260809 02:14:52.954681 9377 webserver.cc:460] Webserver started at https://127.0.0.1:38073/ using document root <none> and password file <none>
I20260809 02:14:52.954825 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:14:52.955039 9377 webserver.cc:460] Webserver started at http://127.0.0.1:44227/ using document root <none> and password file <none>
2026-08-09T02:14:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:52Z Disabled control of system clock
I20260809 02:14:52.964596 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35945
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:44575
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35945
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38073/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/testchainca.cert with env {}
W20260809 02:14:53.052407 15114 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:53.052630 15114 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:53.052654 15114 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:53.053902 15114 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:53.053956 15114 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:53.053968 15114 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:53.053982 15114 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:53.054001 15114 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:14:53.054014 15114 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:14:53.054026 15114 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:38073/jwks.json
I20260809 02:14:53.055559 15114 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:44575
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35945
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35945
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:38073/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15114
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:53.055820 15114 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:53.056016 15114 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:53.061039 15122 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.061152 15120 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.061144 15114 server_base.cc:1034] running on GCE node
W20260809 02:14:53.061043 15119 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.061631 15114 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:53.061944 15114 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:53.063125 15114 hybrid_clock.cc:648] HybridClock initialized: now 1786241693063107 us; error 48 us; skew 500 ppm
I20260809 02:14:53.064513 15114 webserver.cc:460] Webserver started at http://127.9.40.126:39089/ using document root <none> and password file <none>
I20260809 02:14:53.064831 15114 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:53.064904 15114 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:53.065059 15114 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:53.066154 15114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "9e612e1f3159492f8748ed685cdbcb39"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "9c66cb9c668c730638c960139e27266b"
server_key_iv: "cf80606b3242c983e99d35662babb416"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.066524 15114 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "9e612e1f3159492f8748ed685cdbcb39"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "9c66cb9c668c730638c960139e27266b"
server_key_iv: "cf80606b3242c983e99d35662babb416"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.067838 15114 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:53.068564 15128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:53.068776 15114 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:53.068869 15114 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "9e612e1f3159492f8748ed685cdbcb39"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "9c66cb9c668c730638c960139e27266b"
server_key_iv: "cf80606b3242c983e99d35662babb416"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.068964 15114 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:53.099509 15114 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:53.099885 15114 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:53.100050 15114 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:53.104410 15114 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35945
I20260809 02:14:53.104449 15189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35945 every 8 connection(s)
I20260809 02:14:53.104882 15114 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:53.106268 15190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:53.108808 15190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39: Bootstrap starting.
I20260809 02:14:53.109491 15190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:53.109834 15190 log.cc:824] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:53.109965 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15114
I20260809 02:14:53.110090 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:53.110297 9377 external_mini_cluster.cc:1428] Setting key b64ce1b64ca6592c12e34a39b40d0c41
I20260809 02:14:53.110649 15190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39: No bootstrap required, opened a new log
I20260809 02:14:53.112380 15190 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } }
I20260809 02:14:53.112515 15190 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:53.112537 15190 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e612e1f3159492f8748ed685cdbcb39, State: Initialized, Role: FOLLOWER
I20260809 02:14:53.112699 15190 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } }
I20260809 02:14:53.112838 15190 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:53.112864 15190 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:53.112931 15190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:53.113485 15190 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } }
I20260809 02:14:53.113574 15190 leader_election.cc:302] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e612e1f3159492f8748ed685cdbcb39; no voters:
I20260809 02:14:53.113781 15190 leader_election.cc:288] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:53.113813 15195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:53.113920 15195 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [term 1 LEADER]: Becoming Leader. State: Replica: 9e612e1f3159492f8748ed685cdbcb39, State: Running, Role: LEADER
I20260809 02:14:53.114015 15195 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } }
I20260809 02:14:53.114068 15190 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:53.114388 15195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e612e1f3159492f8748ed685cdbcb39. Latest consensus state: current_term: 1 leader_uuid: "9e612e1f3159492f8748ed685cdbcb39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } } }
I20260809 02:14:53.114475 15195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.114403 15196 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e612e1f3159492f8748ed685cdbcb39" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e612e1f3159492f8748ed685cdbcb39" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35945 } } }
I20260809 02:14:53.114673 15196 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.114706 15200 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:53.115260 15200 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:53.116690 15212 mvcc.cc:205] Tried to move back new op lower bound from 7316445975004368896 to 7316445974995521536. Current Snapshot: MvccSnapshot[applied={T|T < 7316445975004368896}]
I20260809 02:14:53.118016 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.110822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33910 (local address 127.9.40.126:35945)
0809 02:14:53.111057 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.111064 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.111094 (+ 30us) server_negotiation.cc:407] Connection header received
0809 02:14:53.112372 (+ 1278us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.112380 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.112447 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.112634 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:53.112940 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.114514 (+ 1574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.115749 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.116349 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.116857 (+ 508us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.117295 (+ 438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:53.117298 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:53.117299 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:53.117522 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:53.117565 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:53.117580 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:53.117654 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:53.117713 (+ 59us) server_negotiation.cc:299] Negotiation successful
0809 02:14:53.117790 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":82,"threads_started":1}
I20260809 02:14:53.118287 15200 catalog_manager.cc:1347] Generated new cluster ID: 14d8dc35b6504c68bf08e9d1fd493eda
I20260809 02:14:53.118341 15200 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:53.118966 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:53.139219 15200 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:53.140105 15200 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:53.148023 15200 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e612e1f3159492f8748ed685cdbcb39: Generated new TSK 0
I20260809 02:14:53.148267 15200 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:14:53.175146 15220 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260809 02:14:53.175328 15220 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260809 02:14:53.175555 15194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.159685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.9.40.126:35945)
0809 02:14:53.159758 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.159761 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.159900 (+ 139us) server_negotiation.cc:407] Connection header received
0809 02:14:53.160012 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.160013 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.160025 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.160087 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:14:53.160275 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.160842 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.161967 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.162147 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.162197 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.162432 (+ 235us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:14:53.175294 (+ 12862us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:14:53.175484 (+ 190us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33922: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":41}
W20260809 02:14:53.175601 15194 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33922: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260809 02:14:53.175954 15219 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:53.159601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35945 (local address 127.0.0.1:33922)
0809 02:14:53.159815 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:53.159827 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:53.159910 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:53.160119 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:53.160120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:53.160122 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:14:53.160201 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.160203 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.160922 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.160923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.161879 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.161882 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.162231 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.162232 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.162300 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.162321 (+ 21us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:14:53.175555 (+ 13234us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0809 02:14:53.175682 (+ 127us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:35945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":113,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.9.40.126:35945: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:38073/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260809 02:14:53.176435 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15114
2026-08-09T02:14:53Z chronyd exiting
I20260809 02:14:53.215561 9377 test_util.cc:175] -----------------------------------------------
I20260809 02:14:53.215682 9377 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786241669736972-9377-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (263 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260809 02:14:53.216535 9377 mini_oidc.cc:152] Starting JWKS server
I20260809 02:14:53.217344 9377 webserver.cc:460] Webserver started at https://127.0.0.1:46561/ using document root <none> and password file <none>
I20260809 02:14:53.217417 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:14:53.217588 9377 webserver.cc:460] Webserver started at http://127.0.0.1:44847/ using document root <none> and password file <none>
2026-08-09T02:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:53Z Disabled control of system clock
I20260809 02:14:53.226673 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39213
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:36363
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39213
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46561/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/testchainca.cert with env {}
W20260809 02:14:53.311443 15227 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:53.311662 15227 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:53.311686 15227 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:53.312868 15227 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:53.312907 15227 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:53.312920 15227 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:53.312940 15227 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:53.312952 15227 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:14:53.312965 15227 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:14:53.312976 15227 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:46561/jwks.json
I20260809 02:14:53.314437 15227 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36363
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39213
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39213
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:46561/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15227
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:53.314640 15227 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:53.314812 15227 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:53.319763 15233 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.319774 15232 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.319747 15235 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.320034 15227 server_base.cc:1034] running on GCE node
I20260809 02:14:53.320300 15227 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:53.320549 15227 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:53.321688 15227 hybrid_clock.cc:648] HybridClock initialized: now 1786241693321649 us; error 53 us; skew 500 ppm
I20260809 02:14:53.322948 15227 webserver.cc:460] Webserver started at http://127.9.40.126:32773/ using document root <none> and password file <none>
I20260809 02:14:53.323242 15227 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:53.323293 15227 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:53.323468 15227 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:53.324542 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "856780f628e5491fa5b322ce354d5750"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "de8bf4484a4ca89a9705d6c3e5e905f9"
server_key_iv: "80f11c92be9e7e45ba57b8057e6a8139"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.324949 15227 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "856780f628e5491fa5b322ce354d5750"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "de8bf4484a4ca89a9705d6c3e5e905f9"
server_key_iv: "80f11c92be9e7e45ba57b8057e6a8139"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.326313 15227 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:53.327246 15241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:53.327528 15227 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:53.327633 15227 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "856780f628e5491fa5b322ce354d5750"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "de8bf4484a4ca89a9705d6c3e5e905f9"
server_key_iv: "80f11c92be9e7e45ba57b8057e6a8139"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.327714 15227 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:53.347651 15227 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:53.348030 15227 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:53.348208 15227 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:53.352216 15227 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39213
I20260809 02:14:53.352299 15300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39213 every 8 connection(s)
I20260809 02:14:53.352630 15227 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:53.353607 15301 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:53.355897 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750: Bootstrap starting.
I20260809 02:14:53.356441 15301 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:53.356688 15301 log.cc:824] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:53.357461 15301 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750: No bootstrap required, opened a new log
I20260809 02:14:53.359091 15301 raft_consensus.cc:348] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } }
I20260809 02:14:53.359242 15301 raft_consensus.cc:374] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:53.359265 15301 raft_consensus.cc:729] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 856780f628e5491fa5b322ce354d5750, State: Initialized, Role: FOLLOWER
I20260809 02:14:53.359416 15301 consensus_queue.cc:260] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } }
I20260809 02:14:53.359499 15301 raft_consensus.cc:388] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:53.359524 15301 raft_consensus.cc:482] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:53.359568 15301 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:53.360092 15301 raft_consensus.cc:504] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } }
I20260809 02:14:53.360188 15301 leader_election.cc:302] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 856780f628e5491fa5b322ce354d5750; no voters:
I20260809 02:14:53.360332 15301 leader_election.cc:288] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:53.360404 15306 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:53.360581 15306 raft_consensus.cc:686] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [term 1 LEADER]: Becoming Leader. State: Replica: 856780f628e5491fa5b322ce354d5750, State: Running, Role: LEADER
I20260809 02:14:53.360648 15301 sys_catalog.cc:564] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:53.360683 15306 consensus_queue.cc:237] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } }
I20260809 02:14:53.361145 15307 sys_catalog.cc:455] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "856780f628e5491fa5b322ce354d5750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } } }
I20260809 02:14:53.361251 15307 sys_catalog.cc:458] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.361299 15308 sys_catalog.cc:455] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 856780f628e5491fa5b322ce354d5750. Latest consensus state: current_term: 1 leader_uuid: "856780f628e5491fa5b322ce354d5750" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "856780f628e5491fa5b322ce354d5750" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39213 } } }
I20260809 02:14:53.361393 15308 sys_catalog.cc:458] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.361485 15314 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:53.361675 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15227
I20260809 02:14:53.361773 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:53.361989 9377 external_mini_cluster.cc:1428] Setting key f4a1de62606682b0bd2ffce9cfc32fd3
I20260809 02:14:53.362015 15314 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:53.366071 15314 catalog_manager.cc:1347] Generated new cluster ID: e023637459384f0db53d74792eed6d0e
I20260809 02:14:53.366142 15314 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:53.368149 15325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.362683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40002 (local address 127.9.40.126:39213)
0809 02:14:53.362859 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.362868 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.362899 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:14:53.363013 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.363019 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.363086 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.363234 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:53.363418 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.364837 (+ 1419us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.365921 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.366453 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.366589 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.367456 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:53.367459 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:53.367460 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:53.367650 (+ 190us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:53.367683 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:53.367695 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:53.367775 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:53.367851 (+ 76us) server_negotiation.cc:299] Negotiation successful
0809 02:14:53.367934 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":80,"threads_started":1}
I20260809 02:14:53.369166 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:53.375715 15314 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:53.376230 15314 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:53.378083 15314 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 856780f628e5491fa5b322ce354d5750: Generated new TSK 0
I20260809 02:14:53.378257 15314 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260809 02:14:53.410362 15335 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260809 02:14:53.410538 15335 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
W20260809 02:14:53.410861 15334 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:53.385873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39213 (local address 127.0.0.1:40018)
0809 02:14:53.386034 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:53.386046 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:53.386116 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:53.386275 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:53.386276 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:53.386278 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:14:53.386344 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.386346 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.387001 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.387002 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.387976 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.387979 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.389368 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.389369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.389437 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.389444 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:14:53.410676 (+ 21232us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0809 02:14:53.410779 (+ 103us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:39213: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:46561/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":95,"threads_started":1}
I20260809 02:14:53.411080 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15227
2026-08-09T02:14:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (235 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260809 02:14:53.451915 9377 mini_oidc.cc:152] Starting JWKS server
I20260809 02:14:53.452663 9377 webserver.cc:460] Webserver started at https://127.0.0.1:36495/ using document root <none> and password file <none>
I20260809 02:14:53.452711 9377 mini_oidc.cc:189] Starting OIDC Discovery server
I20260809 02:14:53.452929 9377 webserver.cc:460] Webserver started at http://127.0.0.1:35007/ using document root <none> and password file <none>
2026-08-09T02:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:53Z Disabled control of system clock
I20260809 02:14:53.461853 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37293
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:41967
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37293
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36495/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:14:53.543242 15343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:53.543462 15343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:53.543483 15343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:53.544584 15343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:53.544622 15343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:53.544636 15343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:53.544647 15343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:53.544659 15343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:14:53.544672 15343 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:14:53.544683 15343 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36495/jwks.json
I20260809 02:14:53.546017 15343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41967
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37293
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36495/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:53.546228 15343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:53.546491 15343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:53.551621 15348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.551626 15351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.551661 15343 server_base.cc:1034] running on GCE node
W20260809 02:14:53.551626 15349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.552166 15343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:53.552497 15343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:53.553661 15343 hybrid_clock.cc:648] HybridClock initialized: now 1786241693553625 us; error 48 us; skew 500 ppm
I20260809 02:14:53.555119 15343 webserver.cc:460] Webserver started at http://127.9.40.126:41371/ using document root <none> and password file <none>
I20260809 02:14:53.555390 15343 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:53.555441 15343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:53.555619 15343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:53.556818 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "f7458f6b1b8a40c883c7a190777ad780"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "109b374b31c94cd7fc453570becf9b7d"
server_key_iv: "bb1510a7618a239b1889f58bc272c3b6"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.557196 15343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "f7458f6b1b8a40c883c7a190777ad780"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "109b374b31c94cd7fc453570becf9b7d"
server_key_iv: "bb1510a7618a239b1889f58bc272c3b6"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.558538 15343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:14:53.559512 15357 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:53.559727 15343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:53.559810 15343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "f7458f6b1b8a40c883c7a190777ad780"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "109b374b31c94cd7fc453570becf9b7d"
server_key_iv: "bb1510a7618a239b1889f58bc272c3b6"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.559865 15343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:53.569481 15343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:53.569772 15343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:53.569880 15343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:53.573951 15343 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37293
I20260809 02:14:53.574038 15412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37293 every 8 connection(s)
I20260809 02:14:53.574374 15343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:53.575074 15413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:53.576385 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15343
I20260809 02:14:53.576470 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:53.576655 9377 external_mini_cluster.cc:1428] Setting key 3ab11d611be366fdd66f1f5a94e5b157
I20260809 02:14:53.578904 15413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780: Bootstrap starting.
I20260809 02:14:53.579602 15413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:53.579905 15413 log.cc:824] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:53.580644 15413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780: No bootstrap required, opened a new log
I20260809 02:14:53.582834 15413 raft_consensus.cc:348] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } }
I20260809 02:14:53.582983 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.577235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45058 (local address 127.9.40.126:37293)
0809 02:14:53.577455 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.577464 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.577496 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:14:53.578047 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.578056 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.578149 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.578304 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:53.578468 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.579798 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.580586 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.581248 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.581380 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.582187 (+ 807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:53.582191 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:53.582191 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:53.582430 (+ 239us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:53.582483 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:53.582500 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:53.582579 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:53.582659 (+ 80us) server_negotiation.cc:299] Negotiation successful
0809 02:14:53.582740 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":86,"threads_started":1}
I20260809 02:14:53.583030 15413 raft_consensus.cc:374] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:53.583069 15413 raft_consensus.cc:729] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f7458f6b1b8a40c883c7a190777ad780, State: Initialized, Role: FOLLOWER
I20260809 02:14:53.583187 15413 consensus_queue.cc:260] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } }
I20260809 02:14:53.583281 15413 raft_consensus.cc:388] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:53.583325 15413 raft_consensus.cc:482] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:53.583374 15413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:53.584043 15413 raft_consensus.cc:504] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } }
I20260809 02:14:53.584165 15413 leader_election.cc:302] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f7458f6b1b8a40c883c7a190777ad780; no voters:
I20260809 02:14:53.584172 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:53.584342 15413 leader_election.cc:288] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:53.584378 15418 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:53.584496 15418 raft_consensus.cc:686] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [term 1 LEADER]: Becoming Leader. State: Replica: f7458f6b1b8a40c883c7a190777ad780, State: Running, Role: LEADER
I20260809 02:14:53.584582 15418 consensus_queue.cc:237] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } }
I20260809 02:14:53.584628 15413 sys_catalog.cc:564] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:53.584923 15418 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f7458f6b1b8a40c883c7a190777ad780. Latest consensus state: current_term: 1 leader_uuid: "f7458f6b1b8a40c883c7a190777ad780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } } }
I20260809 02:14:53.584997 15418 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.585161 15419 sys_catalog.cc:455] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f7458f6b1b8a40c883c7a190777ad780" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f7458f6b1b8a40c883c7a190777ad780" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37293 } } }
I20260809 02:14:53.585239 15419 sys_catalog.cc:458] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.585368 15423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:53.585899 15423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:53.587661 15423 catalog_manager.cc:1347] Generated new cluster ID: e6a6dee27676427cb9bcdb29af6ba0c3
I20260809 02:14:53.587713 15423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:53.606992 15423 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:53.607769 15423 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:53.622608 15423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f7458f6b1b8a40c883c7a190777ad780: Generated new TSK 0
I20260809 02:14:53.622850 15423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:53.632550 15416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.628700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45068 (local address 127.9.40.126:37293)
0809 02:14:53.628779 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.628781 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.628934 (+ 153us) server_negotiation.cc:407] Connection header received
0809 02:14:53.629019 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.629020 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.629031 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.629124 (+ 93us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:14:53.629245 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.629752 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.630886 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.631073 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.631129 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.631362 (+ 233us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:14:53.632338 (+ 976us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:14:53.632472 (+ 134us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45068: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":53}
W20260809 02:14:53.632620 15416 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45068: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260809 02:14:53.632674 15443 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:53.628605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37293 (local address 127.0.0.1:45068)
0809 02:14:53.628844 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:53.628856 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:53.628942 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:53.629115 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:53.629116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:53.629118 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:14:53.629179 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.629182 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.629835 (+ 653us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.629836 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.630796 (+ 960us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.630800 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.631165 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.631166 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.631249 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.631258 (+ 9us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:14:53.632446 (+ 1188us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0809 02:14:53.632557 (+ 111us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:37293: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36495/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":113,"threads_started":1}
I20260809 02:14:53.633302 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15343
E20260809 02:14:53.647354 15444 webserver.cc:533] Webserver: sslize failed:
E20260809 02:14:53.647467 15444 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-09T02:14:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (223 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-09T02:14:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:53Z Disabled control of system clock
I20260809 02:14:53.684248 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35815
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:37135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35815
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/testchainca.cert with env {}
W20260809 02:14:53.766233 15449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:53.766489 15449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:53.766528 15449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:53.767692 15449 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:53.767760 15449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:53.767781 15449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:53.767802 15449 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:53.767833 15449 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260809 02:14:53.767858 15449 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260809 02:14:53.767880 15449 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260809 02:14:53.769423 15449 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35815
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35815
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:53.769727 15449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:53.769951 15449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:53.774988 15457 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.775050 15455 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:53.775015 15454 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:53.775038 15449 server_base.cc:1034] running on GCE node
I20260809 02:14:53.775434 15449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:53.775745 15449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:53.776923 15449 hybrid_clock.cc:648] HybridClock initialized: now 1786241693776900 us; error 35 us; skew 500 ppm
I20260809 02:14:53.778241 15449 webserver.cc:460] Webserver started at http://127.9.40.126:40639/ using document root <none> and password file <none>
I20260809 02:14:53.778506 15449 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:53.778575 15449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:53.778738 15449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:53.779830 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "564b4b1bb60f4ba592a9a557200eeeda"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "07ce7a927a1d1dfb69cd8e53ed49e1dd"
server_key_iv: "da318d6c1578b752dc1570176c187cc3"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.780208 15449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "564b4b1bb60f4ba592a9a557200eeeda"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "07ce7a927a1d1dfb69cd8e53ed49e1dd"
server_key_iv: "da318d6c1578b752dc1570176c187cc3"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.781579 15449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:53.782346 15463 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:53.782521 15449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:14:53.782634 15449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "564b4b1bb60f4ba592a9a557200eeeda"
format_stamp: "Formatted at 2026-08-09 02:14:53 on dist-test-slave-qg60"
server_key: "07ce7a927a1d1dfb69cd8e53ed49e1dd"
server_key_iv: "da318d6c1578b752dc1570176c187cc3"
server_key_version: "encryptionkey@0"
I20260809 02:14:53.782711 15449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:53.795323 15449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:53.795660 15449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:53.795818 15449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:53.800370 15449 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35815
I20260809 02:14:53.800426 15519 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35815 every 8 connection(s)
I20260809 02:14:53.800861 15449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:53.801867 15520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:53.804369 15520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda: Bootstrap starting.
I20260809 02:14:53.805040 15520 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:53.805365 15520 log.cc:824] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:53.806111 15520 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda: No bootstrap required, opened a new log
I20260809 02:14:53.807875 15520 raft_consensus.cc:348] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } }
I20260809 02:14:53.808022 15520 raft_consensus.cc:374] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:53.808075 15520 raft_consensus.cc:729] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 564b4b1bb60f4ba592a9a557200eeeda, State: Initialized, Role: FOLLOWER
I20260809 02:14:53.808248 15520 consensus_queue.cc:260] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } }
I20260809 02:14:53.808338 15520 raft_consensus.cc:388] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:53.808386 15520 raft_consensus.cc:482] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:53.808444 15520 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:53.808951 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15449
I20260809 02:14:53.809084 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:53.809324 9377 external_mini_cluster.cc:1428] Setting key 2de450b8503737d143e7a479c763cbf7
I20260809 02:14:53.809317 15520 raft_consensus.cc:504] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } }
I20260809 02:14:53.809425 15520 leader_election.cc:302] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 564b4b1bb60f4ba592a9a557200eeeda; no voters:
I20260809 02:14:53.809608 15520 leader_election.cc:288] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:53.809685 15523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:53.809837 15523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [term 1 LEADER]: Becoming Leader. State: Replica: 564b4b1bb60f4ba592a9a557200eeeda, State: Running, Role: LEADER
I20260809 02:14:53.809892 15520 sys_catalog.cc:564] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:53.809926 15523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } }
I20260809 02:14:53.810268 15523 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [sys.catalog]: SysCatalogTable state changed. Reason: New leader 564b4b1bb60f4ba592a9a557200eeeda. Latest consensus state: current_term: 1 leader_uuid: "564b4b1bb60f4ba592a9a557200eeeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } } }
I20260809 02:14:53.810271 15524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "564b4b1bb60f4ba592a9a557200eeeda" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "564b4b1bb60f4ba592a9a557200eeeda" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35815 } } }
I20260809 02:14:53.810377 15523 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.810437 15524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:53.810832 15532 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:53.811445 15532 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:53.813635 15532 catalog_manager.cc:1347] Generated new cluster ID: 96b3b652f3a449529a42c69b27a29a4a
I20260809 02:14:53.813696 15532 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:14:53.816074 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.809838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45344 (local address 127.9.40.126:35815)
0809 02:14:53.810067 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.810080 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.810204 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:14:53.810573 (+ 369us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.810581 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.810656 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.810771 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:53.811167 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.812763 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.813922 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.814395 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.814548 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.815313 (+ 765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:53.815317 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:53.815317 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:14:53.815516 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:53.815551 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:14:53.815562 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:53.815665 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:53.815749 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:14:53.815839 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":72,"threads_started":1}
I20260809 02:14:53.817086 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:14:53.843112 15532 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:53.843873 15532 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:53.855453 15532 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 564b4b1bb60f4ba592a9a557200eeeda: Generated new TSK 0
I20260809 02:14:53.855679 15532 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:14:53.954624 15527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:53.862529 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45354 (local address 127.9.40.126:35815)
0809 02:14:53.862599 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:53.862601 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:53.862727 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:14:53.862808 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:53.862809 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:53.862818 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:53.862880 (+ 62us) server_negotiation.cc:226] Negotiated authn=JWT
0809 02:14:53.863027 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.863538 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.864720 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.865003 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.865057 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.865275 (+ 218us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0809 02:14:53.954359 (+ 89084us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:14:53.954535 (+ 176us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45354: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":39}
W20260809 02:14:53.954735 15527 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45354: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260809 02:14:53.954742 15552 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:53.862481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35815 (local address 127.0.0.1:45354)
0809 02:14:53.862647 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:53.862659 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:53.862734 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:53.862897 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:53.862898 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:53.862900 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0809 02:14:53.862961 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.862964 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.863629 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.863630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.864615 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:53.864618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:53.865092 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:53.865094 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:53.865159 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:53.865166 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0809 02:14:53.954514 (+ 89348us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0809 02:14:53.954662 (+ 148us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.9.40.126:35815: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":109,"threads_started":1}
I20260809 02:14:53.955134 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15449
2026-08-09T02:14:53Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (299 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (93 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:54 dist-test-slave-qg60 krb5kdc[15566](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:54 dist-test-slave-qg60 krb5kdc[15566](info): set up 2 sockets
Aug 09 02:14:54 dist-test-slave-qg60 krb5kdc[15566](info): commencing operation
krb5kdc: starting...
W20260809 02:14:55.257146 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.096s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:14:55.384362 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39777
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:39777
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:55.466972 15580 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:55.467235 15580 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:55.467273 15580 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:55.468441 15580 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:55.468519 15580 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:55.468546 15580 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:55.468577 15580 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:55.468598 15580 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:55.470194 15580 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:39777
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:39777
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15580
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:55.470484 15580 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:55.470733 15580 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:55.475875 15588 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:55.475912 15585 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:55.476117 15586 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:55.476118 15580 server_base.cc:1034] running on GCE node
I20260809 02:14:55.476572 15580 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:55.476951 15580 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:55.478134 15580 hybrid_clock.cc:648] HybridClock initialized: now 1786241695478117 us; error 28 us; skew 500 ppm
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:55.480989 15580 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:14:55.481535 15580 webserver.cc:460] Webserver started at http://127.9.40.126:40219/ using document root <none> and password file <none>
I20260809 02:14:55.481798 15580 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:55.481873 15580 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:55.482036 15580 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:55.483193 15580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "190e2f9fc616457187cda405c590a8fd"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "11f51faf805c5906bfc3d744b5be8b39"
server_key_iv: "e7cd582c74d2c6ade0f05e563bd0bdc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.483566 15580 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "190e2f9fc616457187cda405c590a8fd"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "11f51faf805c5906bfc3d744b5be8b39"
server_key_iv: "e7cd582c74d2c6ade0f05e563bd0bdc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.485028 15580 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:55.485874 15596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.486052 15580 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:55.486191 15580 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "190e2f9fc616457187cda405c590a8fd"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "11f51faf805c5906bfc3d744b5be8b39"
server_key_iv: "e7cd582c74d2c6ade0f05e563bd0bdc6"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.486274 15580 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:55.494751 15580 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:55.496606 15580 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:55.496842 15580 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:55.501354 15580 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:39777
I20260809 02:14:55.501410 15652 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:39777 every 8 connection(s)
I20260809 02:14:55.501782 15580 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:55.502656 15653 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:55.505137 15653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd: Bootstrap starting.
I20260809 02:14:55.505774 15653 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:55.506083 15653 log.cc:824] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:55.506816 15653 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd: No bootstrap required, opened a new log
I20260809 02:14:55.508430 15653 raft_consensus.cc:348] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } }
I20260809 02:14:55.508569 15653 raft_consensus.cc:374] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:55.508636 15653 raft_consensus.cc:729] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 190e2f9fc616457187cda405c590a8fd, State: Initialized, Role: FOLLOWER
I20260809 02:14:55.508819 15653 consensus_queue.cc:260] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } }
I20260809 02:14:55.508888 15653 raft_consensus.cc:388] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:55.508934 15653 raft_consensus.cc:482] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:55.508991 15653 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:55.509534 15653 raft_consensus.cc:504] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } }
I20260809 02:14:55.509661 15653 leader_election.cc:302] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 190e2f9fc616457187cda405c590a8fd; no voters:
I20260809 02:14:55.509848 15653 leader_election.cc:288] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:55.509879 15656 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:55.509994 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15580
I20260809 02:14:55.510113 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:55.510147 15653 sys_catalog.cc:564] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:55.510228 15656 raft_consensus.cc:686] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [term 1 LEADER]: Becoming Leader. State: Replica: 190e2f9fc616457187cda405c590a8fd, State: Running, Role: LEADER
I20260809 02:14:55.510319 15656 consensus_queue.cc:237] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } }
I20260809 02:14:55.510366 9377 external_mini_cluster.cc:1428] Setting key 3bdf3585aa76732c95e9fd6e9f94a113
I20260809 02:14:55.510775 15657 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "190e2f9fc616457187cda405c590a8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } } }
I20260809 02:14:55.510874 15657 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:55.511060 15658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 190e2f9fc616457187cda405c590a8fd. Latest consensus state: current_term: 1 leader_uuid: "190e2f9fc616457187cda405c590a8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "190e2f9fc616457187cda405c590a8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 39777 } } }
I20260809 02:14:55.511137 15658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:55.511260 15668 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:55.511802 15668 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:55.514405 15668 catalog_manager.cc:1347] Generated new cluster ID: b9f1cb7dddc34499864d30128f31729b
I20260809 02:14:55.514460 15668 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:55.522162 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:55.510983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34488 (local address 127.9.40.126:39777)
0809 02:14:55.512661 (+ 1678us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:55.512670 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:55.512703 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:14:55.512843 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:55.512850 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:55.512905 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:55.513034 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:55.513960 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.515035 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.515831 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.516271 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.516402 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:55.518794 (+ 2392us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:55.518798 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:55.518798 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:55.518810 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:55.520482 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.520950 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.520953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.520953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.521053 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.521274 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.521275 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.521276 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.521514 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:55.521685 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:55.521799 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 02:14:55.521947 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1631,"thread_start_us":77,"threads_started":1}
I20260809 02:14:55.529395 15668 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:55.529913 15668 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:55.535347 15668 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 190e2f9fc616457187cda405c590a8fd: Generated new TSK 0
I20260809 02:14:55.535526 15668 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:14:55.572074 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:55.675670 15682 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:55.675877 15682 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:55.675901 15682 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:55.677086 15682 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:55.677129 15682 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:55.677156 15682 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:55.678757 15682 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15682
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:55.678956 15682 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:55.679144 15682 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:55.684290 15691 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:55.684294 15687 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:55.684291 15688 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:55.684782 15682 server_base.cc:1034] running on GCE node
I20260809 02:14:55.685012 15682 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:55.685309 15682 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:55.686514 15682 hybrid_clock.cc:648] HybridClock initialized: now 1786241695686498 us; error 27 us; skew 500 ppm
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:55.689318 15682 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:14:55.689824 15682 webserver.cc:460] Webserver started at http://127.9.40.65:46297/ using document root <none> and password file <none>
I20260809 02:14:55.690089 15682 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:55.690169 15682 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:55.690281 15682 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:55.691324 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "652d74c8b5d64ca2abac8bef4362e445"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "ddb7351a8a2fe9de01f1582311017a12"
server_key_iv: "2f5599507431d550be06759c80e90b0d"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.691635 15682 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "652d74c8b5d64ca2abac8bef4362e445"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "ddb7351a8a2fe9de01f1582311017a12"
server_key_iv: "2f5599507431d550be06759c80e90b0d"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.693010 15682 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:55.693979 15698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.694180 15682 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:55.694247 15682 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "652d74c8b5d64ca2abac8bef4362e445"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "ddb7351a8a2fe9de01f1582311017a12"
server_key_iv: "2f5599507431d550be06759c80e90b0d"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.694303 15682 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:55.719437 15682 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:55.721597 15682 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:55.721724 15682 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:55.721943 15682 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:55.722329 15682 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:55.722368 15682 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.722391 15682 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:55.722404 15682 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.728511 15682 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33887
I20260809 02:14:55.728559 15824 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33887 every 8 connection(s)
I20260809 02:14:55.728987 15682 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:55.738615 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:55.729818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60767 (local address 127.9.40.126:39777)
0809 02:14:55.729883 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:55.729885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:55.730114 (+ 229us) server_negotiation.cc:407] Connection header received
0809 02:14:55.731053 (+ 939us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:55.731054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:55.731067 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:55.731133 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:55.731783 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.732287 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.733402 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.733611 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.733687 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:55.735965 (+ 2278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:55.735966 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:55.735966 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:55.735975 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:55.737291 (+ 1316us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.737682 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.737683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.737684 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.737734 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.737998 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.738000 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.738001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.738139 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:55.738215 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:55.738477 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:14:55.738547 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:14:55.738775 15828 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:55.729779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39777 (local address 127.9.40.65:60767)
0809 02:14:55.729977 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:55.730023 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:55.730959 (+ 936us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:55.731215 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:55.731219 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:55.731477 (+ 258us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:55.731703 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:55.731707 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.732372 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.732373 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:55.733311 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:55.733314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.733714 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.733715 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:55.733818 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:55.734430 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:55.734438 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:55.735875 (+ 1437us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:55.737369 (+ 1494us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:55.737376 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:55.737378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:55.737607 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:55.737800 (+ 193us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:55.737800 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:55.737801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:55.737900 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:55.738215 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:55.738220 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:55.738339 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 02:14:55.738426 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:14:55.738550 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260809 02:14:55.739107 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15682
I20260809 02:14:55.739245 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:55.739403 15825 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39777
I20260809 02:14:55.739487 9377 external_mini_cluster.cc:1428] Setting key f79d1f30a005c3f42bdb72093b2b5038
I20260809 02:14:55.739499 15825 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:55.739719 15825 heartbeater.cc:511] Master 127.9.40.126:39777 requested a full tablet report, sending...
I20260809 02:14:55.740324 15612 ts_manager.cc:194] Registered new tserver with Master: 652d74c8b5d64ca2abac8bef4362e445 (127.9.40.65:33887)
I20260809 02:14:55.741027 15612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:60767
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:14:55.768152 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:55.859020 15831 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:55.859326 15831 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:55.859367 15831 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:55.861091 15831 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:55.861172 15831 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:55.861222 15831 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:55.863509 15831 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15831
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:55.863808 15831 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:55.864104 15831 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:55.869864 15837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:55.869899 15836 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:55.869942 15831 server_base.cc:1034] running on GCE node
W20260809 02:14:55.869866 15839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:55.870307 15831 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:55.870581 15831 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:55.871762 15831 hybrid_clock.cc:648] HybridClock initialized: now 1786241695871755 us; error 51 us; skew 500 ppm
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:55.874575 15831 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:14:55.875088 15831 webserver.cc:460] Webserver started at http://127.9.40.66:33687/ using document root <none> and password file <none>
I20260809 02:14:55.875340 15831 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:55.875414 15831 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:55.875571 15831 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:55.876647 15831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "c00f656e7911467bb6944a9ef831b8ce"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "68a33c30ebeaeafae7ae200e45354213"
server_key_iv: "b4038591a56371d91ed3387aca31143b"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.877099 15831 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "c00f656e7911467bb6944a9ef831b8ce"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "68a33c30ebeaeafae7ae200e45354213"
server_key_iv: "b4038591a56371d91ed3387aca31143b"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.878484 15831 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:55.879262 15846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.879444 15831 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:55.879550 15831 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "c00f656e7911467bb6944a9ef831b8ce"
format_stamp: "Formatted at 2026-08-09 02:14:55 on dist-test-slave-qg60"
server_key: "68a33c30ebeaeafae7ae200e45354213"
server_key_iv: "b4038591a56371d91ed3387aca31143b"
server_key_version: "encryptionkey@0"
I20260809 02:14:55.879639 15831 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:55.887105 15831 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:55.889214 15831 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:55.889377 15831 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:55.889659 15831 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:55.889997 15831 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:55.890056 15831 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.890098 15831 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:55.890125 15831 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:55.896080 15831 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:33023
I20260809 02:14:55.896126 15971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:33023 every 8 connection(s)
I20260809 02:14:55.896453 15831 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:14:55 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241695, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:55.904083 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15831
I20260809 02:14:55.904203 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:55.904407 9377 external_mini_cluster.cc:1428] Setting key 4289161ac1c0c0d0cd840a246f1f6839
I20260809 02:14:55.907042 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:55.897252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59277 (local address 127.9.40.126:39777)
0809 02:14:55.897318 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:55.897320 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:55.897551 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:14:55.898696 (+ 1145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:55.898698 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:55.898711 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:55.898778 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:55.899422 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.899970 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.901057 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.901278 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.901373 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:55.903563 (+ 2190us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:55.903564 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:55.903565 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:55.903603 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:55.905520 (+ 1917us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.905958 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.905959 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.905959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.906063 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:55.906346 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:55.906347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:55.906347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:55.906469 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:55.906543 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:55.906851 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:14:55.906942 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:14:55.907261 15974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:55.897225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39777 (local address 127.9.40.66:59277)
0809 02:14:55.897427 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:55.897465 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:55.898576 (+ 1111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:55.898852 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:55.898861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:55.899121 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:55.899318 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:55.899322 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.900081 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.900082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:55.900957 (+ 875us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:55.900959 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:55.901388 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:55.901389 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:55.901513 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:55.902037 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:55.902045 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:55.903473 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:55.905588 (+ 2115us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:55.905596 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:55.905598 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:55.905851 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:55.906144 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:55.906145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:55.906146 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:55.906264 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:55.906548 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:55.906556 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:55.906703 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:14:55.906816 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:14:55.907008 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":97,"threads_started":1}
I20260809 02:14:55.907752 15972 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39777
I20260809 02:14:55.907871 15972 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:55.908085 15972 heartbeater.cc:511] Master 127.9.40.126:39777 requested a full tablet report, sending...
I20260809 02:14:55.908531 15612 ts_manager.cc:194] Registered new tserver with Master: c00f656e7911467bb6944a9ef831b8ce (127.9.40.66:33023)
I20260809 02:14:55.909005 15612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:59277
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:14:55.933140 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:32979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:56.019898 15977 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:56.020157 15977 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:56.020200 15977 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:56.021498 15977 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:56.021575 15977 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:56.021626 15977 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:56.023231 15977 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:32979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:39777
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15977
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:56.023527 15977 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:56.023779 15977 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:56.029441 15984 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:56.029441 15986 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:56.029733 15977 server_base.cc:1034] running on GCE node
W20260809 02:14:56.029477 15983 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:56.030086 15977 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:56.030404 15977 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:56.031594 15977 hybrid_clock.cc:648] HybridClock initialized: now 1786241696031570 us; error 36 us; skew 500 ppm
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.034730 15977 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:14:56.035214 15977 webserver.cc:460] Webserver started at http://127.9.40.67:45389/ using document root <none> and password file <none>
I20260809 02:14:56.035538 15977 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:56.035588 15977 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:56.035776 15977 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:56.037003 15977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "89f5f0bebf7746f79294b20ec4926b95"
format_stamp: "Formatted at 2026-08-09 02:14:56 on dist-test-slave-qg60"
server_key: "9c57ee2198c51f9bfd764d7dcdf1c880"
server_key_iv: "21d6af4273cd9b9aeda22f1b41c3cf18"
server_key_version: "encryptionkey@0"
I20260809 02:14:56.037328 15977 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "89f5f0bebf7746f79294b20ec4926b95"
format_stamp: "Formatted at 2026-08-09 02:14:56 on dist-test-slave-qg60"
server_key: "9c57ee2198c51f9bfd764d7dcdf1c880"
server_key_iv: "21d6af4273cd9b9aeda22f1b41c3cf18"
server_key_version: "encryptionkey@0"
I20260809 02:14:56.038638 15977 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:56.039376 15993 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:56.039621 15977 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:56.039722 15977 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "89f5f0bebf7746f79294b20ec4926b95"
format_stamp: "Formatted at 2026-08-09 02:14:56 on dist-test-slave-qg60"
server_key: "9c57ee2198c51f9bfd764d7dcdf1c880"
server_key_iv: "21d6af4273cd9b9aeda22f1b41c3cf18"
server_key_version: "encryptionkey@0"
I20260809 02:14:56.039817 15977 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:56.054284 15977 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:56.057176 15977 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:56.057339 15977 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:56.057605 15977 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:56.058000 15977 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:56.058061 15977 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:56.058099 15977 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:56.058140 15977 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:56.065634 15977 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38157
I20260809 02:14:56.065701 16119 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38157 every 8 connection(s)
I20260809 02:14:56.066100 15977 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:14:56.070139 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 15977
I20260809 02:14:56.070216 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:56.070412 9377 external_mini_cluster.cc:1428] Setting key b67dc40bb2ef35b1d75c6757e7dbe2aa
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:14:56.076538 15669 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:56.066997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:55961 (local address 127.9.40.126:39777)
0809 02:14:56.067060 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.067062 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.067342 (+ 280us) server_negotiation.cc:407] Connection header received
0809 02:14:56.068597 (+ 1255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.068599 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.068612 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.068664 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.069381 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.069943 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.071387 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.071676 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.071744 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:56.073899 (+ 2155us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:56.073901 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:56.073901 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:56.073909 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:56.075202 (+ 1293us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.075600 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.075601 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.075602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.075659 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.075913 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.075914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.075914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.076037 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:56.076104 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:56.076370 (+ 266us) server_negotiation.cc:299] Negotiation successful
0809 02:14:56.076451 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:14:56.076658 16122 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:56.066940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:39777 (local address 127.9.40.67:55961)
0809 02:14:56.067141 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:56.067193 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:56.068466 (+ 1273us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:56.068751 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:56.068755 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:56.069076 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:56.069274 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:56.069278 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.070068 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.070068 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:56.071307 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:56.071310 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.071753 (+ 443us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.071754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:56.071857 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:56.072332 (+ 475us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:56.072340 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:56.073806 (+ 1466us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:56.075282 (+ 1476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:56.075288 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:56.075290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:56.075508 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:56.075728 (+ 220us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:56.075729 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:56.075729 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:56.075825 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:56.076128 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:56.076132 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:56.076244 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 02:14:56.076328 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:56.076445 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":78,"threads_started":1}
I20260809 02:14:56.077226 16120 heartbeater.cc:347] Connected to a master server at 127.9.40.126:39777
I20260809 02:14:56.077337 16120 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:56.077556 16120 heartbeater.cc:511] Master 127.9.40.126:39777 requested a full tablet report, sending...
I20260809 02:14:56.078109 15612 ts_manager.cc:194] Registered new tserver with Master: 89f5f0bebf7746f79294b20ec4926b95 (127.9.40.67:38157)
I20260809 02:14:56.078511 15612 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:55961
I20260809 02:14:56.082540 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/test-admin.keytab.
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.102837 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260809 02:14:56.104777 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:56.103752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50806 (local address 127.9.40.65:33887)
0809 02:14:56.103947 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.103951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.103962 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:14:56.104013 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.104016 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.104036 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.104119 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.104655 (+ 536us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50806: BlockingRecv error: recv got EOF from 127.0.0.1:50806 (error 108)
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":108,"threads_started":1}
W20260809 02:14:56.104959 9377 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.106030 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.106911 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:56.114933 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:56.107429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50814 (local address 127.9.40.65:33887)
0809 02:14:56.107492 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.107493 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.107502 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:14:56.107563 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.107564 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.107574 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.107621 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.107989 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.108719 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.109578 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.109901 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.109958 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:56.112297 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:56.112305 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:56.112306 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:56.112313 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:56.113853 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.114214 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.114217 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.114217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.114274 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.114453 (+ 179us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.114453 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.114453 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.114585 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:56.114716 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:56.114790 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 02:14:56.114855 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260809 02:14:56.117085 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:56.116340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50818 (local address 127.9.40.65:33887)
0809 02:14:56.116396 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.116397 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.116439 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:14:56.116512 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.116513 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.116524 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.116629 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.117016 (+ 387us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50818: BlockingRecv error: recv got EOF from 127.0.0.1:50818 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:14:56.117252 9377 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.118237 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.119071 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:56.126080 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:56.119539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50824 (local address 127.9.40.65:33887)
0809 02:14:56.119605 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.119608 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.119687 (+ 79us) server_negotiation.cc:407] Connection header received
0809 02:14:56.119784 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.119785 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.119795 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.119849 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.120222 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.120719 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.121448 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.121661 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.121744 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:56.123699 (+ 1955us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:56.123700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:56.123700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:56.123707 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:56.124939 (+ 1232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.125302 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.125303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.125303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.125362 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.125572 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.125573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.125573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.125679 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:56.125784 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:56.125913 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:14:56.125986 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
W20260809 02:14:56.127686 16127 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:14:56.127155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50836 (local address 127.9.40.65:33887)
0809 02:14:56.127197 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.127199 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.127209 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:56.127281 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.127282 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.127292 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.127364 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.127655 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50836: BlockingRecv error: recv got EOF from 127.0.0.1:50836 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:14:56.127895 9377 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786241669736972-9377-0/krb5kdc/krb5cc' of type FILE
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.128904 9377 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:56.129868 9377 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[15566](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:14:56.136920 16127 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:56.130383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50844 (local address 127.9.40.65:33887)
0809 02:14:56.130539 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:56.130541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:56.130596 (+ 55us) server_negotiation.cc:407] Connection header received
0809 02:14:56.130684 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:56.130685 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:56.130697 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:56.130769 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:56.131142 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.131588 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.132274 (+ 686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:56.132463 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:56.132546 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:56.134530 (+ 1984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:56.134531 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:56.134531 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:56.134537 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:56.135943 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.136259 (+ 316us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.136259 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.136260 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.136309 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:56.136475 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:56.136475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:56.136476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:56.136561 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:56.136632 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:56.136793 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:14:56.136857 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260809 02:14:56.137614 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15682
I20260809 02:14:56.146561 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15831
I20260809 02:14:56.152643 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15977
I20260809 02:14:56.158625 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 15580
2026-08-09T02:14:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2019 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[16141](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[16141](info): set up 2 sockets
Aug 09 02:14:56 dist-test-slave-qg60 krb5kdc[16141](info): commencing operation
krb5kdc: starting...
W20260809 02:14:57.318375 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
W20260809 02:14:57.479106 9377 mini_kdc.cc:253] Time spent creating user principal test-admin: real 0.161s user 0.001s sys 0.002s
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260809 02:14:57.616497 9377 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.137s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260809 02:14:57.749753 9377 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.133s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/test-user.keytab.
W20260809 02:14:58.283021 9377 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.533s user 0.000s sys 0.002s
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:14:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:14:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
I20260809 02:14:58.348632 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37597
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37597
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:58.435674 16155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:58.435946 16155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:58.435988 16155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:58.437214 16155 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:58.437297 16155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:58.437325 16155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:58.437353 16155 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:58.437386 16155 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:58.438928 16155 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37597
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:58.439244 16155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:58.439495 16155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:58.444773 16161 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:58.444773 16160 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:58.444792 16163 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:58.445058 16155 server_base.cc:1034] running on GCE node
I20260809 02:14:58.445333 16155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:58.445645 16155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:58.446825 16155 hybrid_clock.cc:648] HybridClock initialized: now 1786241698446815 us; error 51 us; skew 500 ppm
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:58.454780 16155 init.cc:377] Logged in from keytab as oryx/127.9.40.126@KRBTEST.COM (short username oryx)
I20260809 02:14:58.455216 16155 webserver.cc:460] Webserver started at http://127.9.40.126:43813/ using document root <none> and password file <none>
I20260809 02:14:58.455408 16155 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:58.455447 16155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:58.455538 16155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:58.456614 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "dd39532293ad4e55ae39a0f51b8d1606"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "5e4ab4505d4e3b46a409dc721944f076"
server_key_iv: "43177538744241dfc309d9d49534e1ab"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.457007 16155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "dd39532293ad4e55ae39a0f51b8d1606"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "5e4ab4505d4e3b46a409dc721944f076"
server_key_iv: "43177538744241dfc309d9d49534e1ab"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.458346 16155 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:58.459128 16171 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.459318 16155 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:58.459411 16155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "dd39532293ad4e55ae39a0f51b8d1606"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "5e4ab4505d4e3b46a409dc721944f076"
server_key_iv: "43177538744241dfc309d9d49534e1ab"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.459470 16155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:58.469352 16155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:58.471402 16155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:58.471518 16155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:58.475696 16155 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37597
I20260809 02:14:58.475790 16231 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37597 every 8 connection(s)
I20260809 02:14:58.476130 16155 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:14:58.477147 16232 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:58.479422 16232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Bootstrap starting.
I20260809 02:14:58.480046 16232 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:58.480357 16232 log.cc:824] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:58.481149 16232 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: No bootstrap required, opened a new log
I20260809 02:14:58.482775 16232 raft_consensus.cc:348] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } }
I20260809 02:14:58.482944 16232 raft_consensus.cc:374] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:58.482991 16232 raft_consensus.cc:729] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dd39532293ad4e55ae39a0f51b8d1606, State: Initialized, Role: FOLLOWER
I20260809 02:14:58.483193 16232 consensus_queue.cc:260] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } }
I20260809 02:14:58.483301 16232 raft_consensus.cc:388] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:14:58.483342 16232 raft_consensus.cc:482] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:14:58.483395 16232 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:58.484273 16232 raft_consensus.cc:504] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } }
I20260809 02:14:58.484426 16232 leader_election.cc:302] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dd39532293ad4e55ae39a0f51b8d1606; no voters:
I20260809 02:14:58.484652 16232 leader_election.cc:288] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:14:58.484683 16235 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:14:58.484761 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16155
I20260809 02:14:58.484845 16235 raft_consensus.cc:686] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 LEADER]: Becoming Leader. State: Replica: dd39532293ad4e55ae39a0f51b8d1606, State: Running, Role: LEADER
I20260809 02:14:58.484884 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:14:58.484957 16232 sys_catalog.cc:564] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:58.485009 16235 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } }
I20260809 02:14:58.485092 9377 external_mini_cluster.cc:1428] Setting key 74609e7a7764116c8e23f658336eda5c
I20260809 02:14:58.485376 16236 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } }
I20260809 02:14:58.485584 16237 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd39532293ad4e55ae39a0f51b8d1606. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } }
I20260809 02:14:58.485737 16237 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:58.485924 16236 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:58.485970 16246 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:14:58.486828 16246 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:14:58.488631 16246 catalog_manager.cc:1347] Generated new cluster ID: df6bed2e7236435094129a9bfbaea2ee
I20260809 02:14:58.488685 16246 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:58.498742 16246 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:14:58.498970 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:58.485564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45642 (local address 127.9.40.126:37597)
0809 02:14:58.486118 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:58.486130 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:58.486725 (+ 595us) server_negotiation.cc:407] Connection header received
0809 02:14:58.487138 (+ 413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:58.487146 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:58.487216 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:58.487391 (+ 175us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:58.488062 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.489520 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.490736 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.491321 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.491415 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:58.494487 (+ 3072us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:58.494493 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:58.494494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:58.494509 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:58.496835 (+ 2326us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.497261 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.497265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.497265 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.497400 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.497660 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.497661 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.497662 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.498038 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:58.498227 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:58.498317 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:14:58.498683 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":477,"thread_start_us":84,"threads_started":1}
I20260809 02:14:58.499439 16246 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:14:58.519522 16246 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Generated new TSK 0
I20260809 02:14:58.519814 16246 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
I20260809 02:14:58.557723 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:58.641813 16262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:58.642067 16262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:58.642107 16262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:58.643308 16262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:58.643386 16262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:58.643433 16262 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:58.644980 16262 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:58.645282 16262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:58.645534 16262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:58.651131 16267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:58.651265 16262 server_base.cc:1034] running on GCE node
W20260809 02:14:58.651131 16270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:58.651151 16268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:58.651649 16262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:58.651922 16262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:58.653102 16262 hybrid_clock.cc:648] HybridClock initialized: now 1786241698653067 us; error 43 us; skew 500 ppm
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:58.655638 16262 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260809 02:14:58.656077 16262 webserver.cc:460] Webserver started at http://127.9.40.65:40047/ using document root <none> and password file <none>
I20260809 02:14:58.656323 16262 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:58.656394 16262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:58.656553 16262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:58.657725 16262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "d5d452ec7a4940c39cea6b49ab41a406"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "37c26a555ab7804a21c6ea01bf5f7417"
server_key_iv: "473269232d820dad8a3bb888de41f085"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.658138 16262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "d5d452ec7a4940c39cea6b49ab41a406"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "37c26a555ab7804a21c6ea01bf5f7417"
server_key_iv: "473269232d820dad8a3bb888de41f085"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.659504 16262 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:14:58.660283 16277 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.660488 16262 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:58.660588 16262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "d5d452ec7a4940c39cea6b49ab41a406"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "37c26a555ab7804a21c6ea01bf5f7417"
server_key_iv: "473269232d820dad8a3bb888de41f085"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.660684 16262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:58.681757 16262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:58.683844 16262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:58.684032 16262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:58.684316 16262 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:58.684656 16262 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:58.684716 16262 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.684793 16262 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:58.684834 16262 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.690970 16262 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:40255
I20260809 02:14:58.691054 16401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:40255 every 8 connection(s)
I20260809 02:14:58.691398 16262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:58.693964 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16262
I20260809 02:14:58.694058 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:14:58.694303 9377 external_mini_cluster.cc:1428] Setting key 1de8407f709daa600becc02b95755e3d
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:58.703327 16405 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:58.692163 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:48591)
0809 02:14:58.692403 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:58.692457 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:58.693127 (+ 670us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:58.693403 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:58.693407 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:58.694189 (+ 782us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:58.694469 (+ 280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:58.694475 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.695638 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.695639 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:58.696592 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:58.696596 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.697171 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.697173 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:58.697349 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:58.697980 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:58.697995 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:58.699779 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:58.701912 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:58.701919 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:58.701921 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:58.702154 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:58.702399 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:58.702399 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:58.702400 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:58.702500 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:58.702823 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:58.702827 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:58.702946 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 02:14:58.703030 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:14:58.703159 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":94,"threads_started":1}
I20260809 02:14:58.703336 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:58.692195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:48591 (local address 127.9.40.126:37597)
0809 02:14:58.692356 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:58.692359 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:58.692593 (+ 234us) server_negotiation.cc:407] Connection header received
0809 02:14:58.693245 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:58.693246 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:58.693264 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:58.693330 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:58.694600 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.695508 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.696765 (+ 1257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.697070 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.697134 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:58.699899 (+ 2765us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:58.699902 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:58.699902 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:58.699912 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:58.701818 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.702245 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.702246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.702247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.702317 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.702575 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.702576 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.702576 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.702747 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:58.702831 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:58.703094 (+ 263us) server_negotiation.cc:299] Negotiation successful
0809 02:14:58.703197 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":122}
I20260809 02:14:58.704133 16403 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:14:58.704249 16403 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:58.704444 16403 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:14:58.705055 16189 ts_manager.cc:194] Registered new tserver with Master: d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:14:58.705729 16189 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:48591
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
I20260809 02:14:58.729283 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:58.817811 16409 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:58.818022 16409 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:58.818044 16409 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:58.819195 16409 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:58.819242 16409 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:58.819268 16409 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:58.820776 16409 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16409
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:58.821038 16409 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:58.821282 16409 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:58.826311 16414 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:58.826357 16415 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:58.826311 16418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:58.826562 16409 server_base.cc:1034] running on GCE node
I20260809 02:14:58.826767 16409 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:58.827023 16409 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:58.828207 16409 hybrid_clock.cc:648] HybridClock initialized: now 1786241698828179 us; error 35 us; skew 500 ppm
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:58.830967 16409 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260809 02:14:58.831374 16409 webserver.cc:460] Webserver started at http://127.9.40.66:43343/ using document root <none> and password file <none>
I20260809 02:14:58.831569 16409 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:58.831614 16409 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:58.831705 16409 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:58.832685 16409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "fa991920d9a247419c0a95b7afeca1ca"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "914d11e5f69940f08054e80ce55e58d9"
server_key_iv: "1c88a23cd148270dadcec764a6a74e04"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.833077 16409 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "fa991920d9a247419c0a95b7afeca1ca"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "914d11e5f69940f08054e80ce55e58d9"
server_key_iv: "1c88a23cd148270dadcec764a6a74e04"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.834408 16409 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:58.835212 16426 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.835371 16409 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:58.835464 16409 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "fa991920d9a247419c0a95b7afeca1ca"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "914d11e5f69940f08054e80ce55e58d9"
server_key_iv: "1c88a23cd148270dadcec764a6a74e04"
server_key_version: "encryptionkey@0"
I20260809 02:14:58.835523 16409 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:58.850744 16409 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:58.852770 16409 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:58.852896 16409 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:58.853132 16409 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:58.853416 16409 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:58.853447 16409 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.853470 16409 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:58.853482 16409 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:58.859504 16409 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35527
I20260809 02:14:58.859589 16550 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35527 every 8 connection(s)
I20260809 02:14:58.859929 16409 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:58.865129 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16409
I20260809 02:14:58.865257 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:14:58.865726 9377 external_mini_cluster.cc:1428] Setting key bb673bcfdcb36adaaa7ec226cf7472f3
Aug 09 02:14:58 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:58.870400 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:58.860868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35139 (local address 127.9.40.126:37597)
0809 02:14:58.860948 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:58.860950 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:58.861224 (+ 274us) server_negotiation.cc:407] Connection header received
0809 02:14:58.861789 (+ 565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:58.861790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:58.861801 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:58.861858 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:58.862948 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.863499 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.864516 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.864865 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.864958 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:58.867310 (+ 2352us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:58.867311 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:58.867311 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:58.867319 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:58.868871 (+ 1552us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.869363 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.869364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.869365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.869432 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:58.869705 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:58.869706 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:58.869706 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:58.869834 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:58.869916 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:58.870215 (+ 299us) server_negotiation.cc:299] Negotiation successful
0809 02:14:58.870301 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:14:58.870695 16553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:58.860823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:35139)
0809 02:14:58.861092 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:58.861132 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:58.861694 (+ 562us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:58.861928 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:58.861931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:58.862603 (+ 672us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:58.862863 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:58.862868 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.863625 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.863626 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:58.864427 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:58.864429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:58.864985 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:58.864985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:58.865097 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:58.865611 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:58.865619 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:58.867191 (+ 1572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:58.868998 (+ 1807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:58.869007 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:58.869009 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:58.869262 (+ 253us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:58.869504 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:58.869504 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:58.869505 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:58.869628 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:58.869908 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:58.869916 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:58.870069 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 02:14:58.870193 (+ 124us) client_negotiation.cc:240] Negotiation successful
0809 02:14:58.870366 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"thread_start_us":101,"threads_started":1}
I20260809 02:14:58.871300 16551 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:14:58.871424 16551 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:58.871618 16551 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:14:58.872059 16189 ts_manager.cc:194] Registered new tserver with Master: fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527)
I20260809 02:14:58.872447 16189 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35139
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260809 02:14:58.896414 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:58.993086 16559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:58.993319 16559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:58.993350 16559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:58.994704 16559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:58.994760 16559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:58.994791 16559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:58.996508 16559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37597
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:58.996855 16559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:58.997149 16559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.002962 16564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.002959 16567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.003053 16565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.003085 16559 server_base.cc:1034] running on GCE node
I20260809 02:14:59.003412 16559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.003705 16559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.004892 16559 hybrid_clock.cc:648] HybridClock initialized: now 1786241699004880 us; error 31 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.007592 16559 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260809 02:14:59.008067 16559 webserver.cc:460] Webserver started at http://127.9.40.67:43645/ using document root <none> and password file <none>
I20260809 02:14:59.008342 16559 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.008419 16559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.008571 16559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:59.009778 16559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "8ddcde33c86445df9da586dc95035fae"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "5b1add006d5cb056f76d9763c050a10e"
server_key_iv: "69147d0829888361255f2822452e6030"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.010180 16559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "8ddcde33c86445df9da586dc95035fae"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "5b1add006d5cb056f76d9763c050a10e"
server_key_iv: "69147d0829888361255f2822452e6030"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.011732 16559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:14:59.012617 16575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.012904 16559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.013015 16559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "8ddcde33c86445df9da586dc95035fae"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "5b1add006d5cb056f76d9763c050a10e"
server_key_iv: "69147d0829888361255f2822452e6030"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.013103 16559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.033365 16559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.035563 16559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:59.035761 16559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.036037 16559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:59.036418 16559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:59.036480 16559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.036525 16559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:59.036551 16559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.043275 16559 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35429
I20260809 02:14:59.043409 16702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35429 every 8 connection(s)
I20260809 02:14:59.043743 16559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:59.053577 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.044789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60087 (local address 127.9.40.126:37597)
0809 02:14:59.044848 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.044850 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.045039 (+ 189us) server_negotiation.cc:407] Connection header received
0809 02:14:59.045586 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.045587 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.045599 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.045656 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.046626 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.047135 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.048229 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.048415 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.048472 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.050693 (+ 2221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.050694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.050695 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.050702 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.052192 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.052580 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.052581 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.052581 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.052637 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.052951 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.052952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.052952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.053064 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.053149 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.053434 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.053507 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:14:59.053647 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16559
I20260809 02:14:59.053727 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:14:59.053838 16707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.044733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:60087)
0809 02:14:59.044929 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.044968 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.045467 (+ 499us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.045720 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.045723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.046321 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.046521 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.046525 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.047353 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.047354 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.048147 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.048148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.048497 (+ 349us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.048498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.048595 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.049167 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.049176 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.050588 (+ 1412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.052281 (+ 1693us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.052287 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.052289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.052507 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.052702 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.052702 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.052703 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.052887 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.053169 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.053174 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.053291 (+ 117us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.053369 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.053563 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":92,"threads_started":1}
I20260809 02:14:59.053931 9377 external_mini_cluster.cc:1428] Setting key 7130f72a47769a7cdd47bd49ea7a8b24
I20260809 02:14:59.054505 16703 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:14:59.054625 16703 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.054865 16703 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:14:59.055261 16188 ts_manager.cc:194] Registered new tserver with Master: 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:14:59.055635 16188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60087
I20260809 02:14:59.066150 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.125.keytab.
Entry for principal oryx/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.125.keytab.
I20260809 02:14:59.096416 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:33343
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.125.keytab
--principal=oryx/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37597,127.9.40.125:33343
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:59.181186 16710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:59.181406 16710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:59.181429 16710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:59.182641 16710 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:59.182688 16710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:59.182703 16710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:59.182716 16710 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:59.182729 16710 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:59.184149 16710 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37597,127.9.40.125:33343
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.125.keytab
--principal=oryx/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:33343
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
W20260809 02:14:59.184248 16710 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.9.40.126:37597,127.9.40.125:33343'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260809 02:14:59.184360 16710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:59.184527 16710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.189599 16715 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.189670 16716 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.189796 16718 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.189817 16710 server_base.cc:1034] running on GCE node
I20260809 02:14:59.190074 16710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.190405 16710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.191579 16710 hybrid_clock.cc:648] HybridClock initialized: now 1786241699191559 us; error 50 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.194383 16710 init.cc:377] Logged in from keytab as oryx/127.9.40.125@KRBTEST.COM (short username oryx)
I20260809 02:14:59.194868 16710 webserver.cc:460] Webserver started at http://127.9.40.125:43925/ using document root <none> and password file <none>
I20260809 02:14:59.195083 16710 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.195128 16710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.195240 16710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:59.196265 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/instance:
uuid: "45fbacfb7e304be99f48e7d0c9660e63"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "21603e0fe5667051a678912216421446"
server_key_iv: "f5cc7904712f2e6f17fd708b529ee880"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.196596 16710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal/instance:
uuid: "45fbacfb7e304be99f48e7d0c9660e63"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "21603e0fe5667051a678912216421446"
server_key_iv: "f5cc7904712f2e6f17fd708b529ee880"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.198066 16710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.198829 16726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.199033 16710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.199108 16710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "45fbacfb7e304be99f48e7d0c9660e63"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "21603e0fe5667051a678912216421446"
server_key_iv: "f5cc7904712f2e6f17fd708b529ee880"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.199208 16710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.214866 16710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.217360 16710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:59.217535 16710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.222128 16710 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:33343
I20260809 02:14:59.222164 16780 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:33343 every 8 connection(s)
I20260809 02:14:59.222592 16710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
I20260809 02:14:59.223755 16781 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:59.225490 16781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:59.232620 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16710
I20260809 02:14:59.234937 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.226275 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45658 (local address 127.9.40.126:37597)
0809 02:14:59.226366 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.226369 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.226565 (+ 196us) server_negotiation.cc:407] Connection header received
0809 02:14:59.227021 (+ 456us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.227022 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.227033 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.227093 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.227740 (+ 647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.228255 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.229324 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.229508 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.229565 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.231736 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.231737 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.231738 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.231745 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.233270 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.233811 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.233812 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.233813 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.233875 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.234192 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.234192 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.234193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.234405 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.234479 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.234755 (+ 276us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.234841 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:14:59.235039 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.226217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45658)
0809 02:14:59.226439 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.226477 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.226928 (+ 451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.227193 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.227196 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.227450 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.227648 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.227651 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.228343 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.228344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.229246 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.229249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.229586 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.229587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.229686 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.230210 (+ 524us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.230218 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.231642 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.233386 (+ 1744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.233394 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.233396 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.233729 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.233951 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.233952 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.233952 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.234114 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.234488 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.234493 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.234616 (+ 123us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.234707 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.234839 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":122,"threads_started":1}
I20260809 02:14:59.235741 16781 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:14:59.242846 16785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.233217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36276 (local address 127.9.40.125:33343)
0809 02:14:59.233435 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.233439 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.233456 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:14:59.233482 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.233484 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.233496 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.233555 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.234324 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.235328 (+ 1004us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.236439 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.236864 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.236939 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.239748 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.239757 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.239758 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.239765 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.241530 (+ 1765us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.241956 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.241959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.241959 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.242029 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.242245 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.242246 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.242247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.242390 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.242552 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.242637 (+ 85us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.242720 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":107,"threads_started":1}
I20260809 02:14:59.243799 16783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.236003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36284)
0809 02:14:59.236055 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.236066 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.236112 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.236310 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.236311 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.236467 (+ 156us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.236530 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.236532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.237409 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.237412 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.238282 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.238286 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.238733 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.238735 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.238820 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.239249 (+ 429us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.239254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.240467 (+ 1213us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.242768 (+ 2301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.242769 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.242770 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.243002 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.243201 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.243202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.243202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.243268 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.243527 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.243527 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.243574 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.243664 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.243741 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:14:59.243912 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.236080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36284 (local address 127.9.40.125:33343)
0809 02:14:59.236203 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.236205 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.236209 (+ 4us) server_negotiation.cc:407] Connection header received
0809 02:14:59.236221 (+ 12us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.236221 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.236228 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.236273 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.236613 (+ 340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.237328 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.238376 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.238585 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.238730 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.240577 (+ 1847us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.240578 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.240578 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.240585 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.242661 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.243069 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.243070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.243071 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.243137 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.243334 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.243335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.243335 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.243426 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.243517 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.243692 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.243791 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":30,"server-negotiator.queue_time_us":110,"thread_start_us":85,"threads_started":1}
I20260809 02:14:59.245211 16781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Bootstrap starting.
I20260809 02:14:59.245841 16781 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:59.246130 16781 log.cc:824] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:59.246877 16781 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: No bootstrap required, opened a new log
I20260809 02:14:59.248337 16781 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } }
I20260809 02:14:59.248457 16781 raft_consensus.cc:374] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:59.248478 16781 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45fbacfb7e304be99f48e7d0c9660e63, State: Initialized, Role: FOLLOWER
I20260809 02:14:59.248641 16781 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } }
I20260809 02:14:59.248981 16781 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:59.248943 16788 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } } }
I20260809 02:14:59.249063 16788 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:59.250629 16799 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:59.250723 16799 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:59.252398 16710 master_runner.cc:428] Detected that this master 45fbacfb7e304be99f48e7d0c9660e63 is joining an existing cluster
I20260809 02:14:59.252477 16710 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.40.125:33343
I20260809 02:14:59.260095 16809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.253725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36292)
0809 02:14:59.253973 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.253981 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.254048 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.254260 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.254261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.254439 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.254480 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.254482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.255122 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.255123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.255679 (+ 556us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.255681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.256084 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.256085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.256155 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.256635 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.256640 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.257141 (+ 501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.258984 (+ 1843us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.258985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.258985 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.259165 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.259379 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259380 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.259380 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.259439 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.259746 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.259746 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.259785 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.259872 (+ 87us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.259984 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":69,"threads_started":1}
I20260809 02:14:59.260349 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.253770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36292 (local address 127.9.40.125:33343)
0809 02:14:59.253805 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.253807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.254038 (+ 231us) server_negotiation.cc:407] Connection header received
0809 02:14:59.254189 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.254190 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.254200 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.254255 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.254564 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.255034 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.255782 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.255999 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.256074 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.257221 (+ 1147us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.257222 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.257223 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.257229 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.258883 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259250 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.259251 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.259251 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.259304 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259510 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.259511 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.259511 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.259640 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.259722 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.259901 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.260273 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":289,"server-negotiator.queue_time_us":19}
I20260809 02:14:59.261034 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.253594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45660 (local address 127.9.40.126:37597)
0809 02:14:59.253647 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.253649 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.253823 (+ 174us) server_negotiation.cc:407] Connection header received
0809 02:14:59.253850 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.253851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.253866 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.253914 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.254358 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.255122 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.256220 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.256513 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.256555 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.257669 (+ 1114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.257670 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.257670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.257678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.259307 (+ 1629us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259701 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.259702 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.259703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.259765 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259978 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.259979 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.259980 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.260109 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.260191 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.260855 (+ 664us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.260940 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:14:59.261052 16807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.253484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45660)
0809 02:14:59.253658 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.253671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.253736 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.253943 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.253944 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.254158 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.254252 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.254255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.255236 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.255236 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.256100 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.256103 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.256634 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.256635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.256708 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.257173 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.257178 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.257560 (+ 382us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.259406 (+ 1846us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259407 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.259407 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.259619 (+ 212us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.259848 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.259849 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.259849 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.259919 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.260586 (+ 667us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.260586 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.260667 (+ 81us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.260817 (+ 150us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.260948 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":110,"threads_started":1}
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:14:59.270915 16786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.263747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36296 (local address 127.9.40.125:33343)
0809 02:14:59.263784 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.263786 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.263936 (+ 150us) server_negotiation.cc:407] Connection header received
0809 02:14:59.264028 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.264029 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.264040 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.264096 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.264512 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.265122 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.265892 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.266095 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.266183 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.268332 (+ 2149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.268334 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.268334 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.268341 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.269688 (+ 1347us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.270090 (+ 402us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.270092 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.270092 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.270157 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.270367 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.270368 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.270369 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.270474 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.270547 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.270738 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.270817 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 02:14:59.270938 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.263688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36296)
0809 02:14:59.263824 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.263872 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.263962 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.264100 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.264103 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.264379 (+ 276us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.264427 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.264429 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.265214 (+ 785us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.265215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.265801 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.265803 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.266183 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.266184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.266274 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.266715 (+ 441us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.266719 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.268237 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.269789 (+ 1552us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.269796 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.269798 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.269995 (+ 197us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.270236 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.270237 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.270237 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.270297 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.270551 (+ 254us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.270559 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.270610 (+ 51us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.270711 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.270826 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":75,"threads_started":1}
I20260809 02:14:59.271765 16188 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.40.125:33343: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63"
member_type: NON_VOTER
last_known_addr {
host: "127.9.40.125"
port: 33343
}
attrs {
promote: true
}
}
dest_uuid: "dd39532293ad4e55ae39a0f51b8d1606"
cas_config_opid_index: -1
I20260809 02:14:59.272074 16188 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } }
I20260809 02:14:59.272550 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } }
I20260809 02:14:59.272897 16755 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:59.273765 16811 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.40.125:33343
I20260809 02:14:59.273859 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } }
I20260809 02:14:59.273900 16755 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260809 02:14:59.273926 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:59.274147 16710 master.cc:537] Master@127.9.40.125:33343 shutting down...
I20260809 02:14:59.274389 16812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:14:59.275104 16173 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260809 02:14:59.275535 16173 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 -> Peer 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125:33343): Couldn't send request to peer 45fbacfb7e304be99f48e7d0c9660e63. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:14:59.277462 16710 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:59.277565 16710 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:59.277618 16710 tablet_replica.cc:330] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: stopping tablet replica
I20260809 02:14:59.290512 16710 master.cc:552] Master@127.9.40.125:33343 shutdown complete.
I20260809 02:14:59.290591 16710 master_runner.cc:305] Clearing existing system tablet
I20260809 02:14:59.290746 16710 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.290815 16710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.291462 16710 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:14:59.291973 16817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.292169 16710 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.292243 16710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:59.292268 16710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "45fbacfb7e304be99f48e7d0c9660e63"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "21603e0fe5667051a678912216421446"
server_key_iv: "f5cc7904712f2e6f17fd708b529ee880"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.293216 16710 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:59.294983 16710 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:14:59.295065 16710 log.cc:1196] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260809 02:14:59.295282 16710 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Deleting consensus metadata
I20260809 02:14:59.295419 16710 master_runner.cc:315] Copying system tablet from 127.9.40.126:37597
I20260809 02:14:59.296077 16710 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.126:37597
I20260809 02:14:59.302981 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.296465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45672 (local address 127.9.40.126:37597)
0809 02:14:59.296518 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.296520 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.296584 (+ 64us) server_negotiation.cc:407] Connection header received
0809 02:14:59.296655 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.296656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.296666 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.296719 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.297135 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.297625 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.298716 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.298894 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.298946 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.300236 (+ 1290us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.300238 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.300238 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.300244 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.301745 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.302265 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.302265 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.302266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.302318 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.302523 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.302524 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.302524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.302635 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.302687 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.302847 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.302915 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:14:59.303069 16823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.296356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45672)
0809 02:14:59.296501 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.296513 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.296588 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.296749 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.296750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.296970 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.297041 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.297044 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.297713 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.297714 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.298628 (+ 914us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.298631 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.299033 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.299034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.299107 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.299668 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.299675 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.300144 (+ 469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.301834 (+ 1690us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.301835 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.301836 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.302182 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.302388 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.302389 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.302390 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.302460 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.302703 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.302704 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.302753 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.302832 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.302950 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":98,"threads_started":1}
I20260809 02:14:59.303468 16211 tablet_copy_service.cc:140] P dd39532293ad4e55ae39a0f51b8d1606: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 45fbacfb7e304be99f48e7d0c9660e63 ({username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:45672)
I20260809 02:14:59.303581 16211 tablet_copy_service.cc:161] P dd39532293ad4e55ae39a0f51b8d1606: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 45fbacfb7e304be99f48e7d0c9660e63 at {username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:45672: session id = 45fbacfb7e304be99f48e7d0c9660e63-00000000000000000000000000000000
I20260809 02:14:59.304461 16211 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:14:59.305226 16710 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:14:59.306217 16710 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: tablet copy: Starting download of 0 data blocks...
I20260809 02:14:59.306401 16710 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: tablet copy: Starting download of 1 WAL segments...
I20260809 02:14:59.307433 16710 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:14:59.308267 16211 tablet_copy_service.cc:342] P dd39532293ad4e55ae39a0f51b8d1606: Request end of tablet copy session 45fbacfb7e304be99f48e7d0c9660e63-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.40.125@KRBTEST.COM'} at 127.0.0.1:45672
I20260809 02:14:59.308346 16211 tablet_copy_service.cc:433] P dd39532293ad4e55ae39a0f51b8d1606: ending tablet copy session 45fbacfb7e304be99f48e7d0c9660e63-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 45fbacfb7e304be99f48e7d0c9660e63
I20260809 02:14:59.310892 16710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.312165 16824 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.312318 16825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.312376 16829 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.312534 16710 server_base.cc:1034] running on GCE node
I20260809 02:14:59.312626 16710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.312784 16710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.313928 16710 hybrid_clock.cc:648] HybridClock initialized: now 1786241699313920 us; error 25 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.315354 16710 init.cc:377] Logged in from keytab as oryx/127.9.40.125@KRBTEST.COM (short username oryx)
I20260809 02:14:59.315659 16710 webserver.cc:460] Webserver started at http://127.9.40.125:33361/ using document root <none> and password file <none>
I20260809 02:14:59.315785 16710 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.315835 16710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.316480 16710 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:14:59.317178 16837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.317309 16710 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.317358 16710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "45fbacfb7e304be99f48e7d0c9660e63"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "21603e0fe5667051a678912216421446"
server_key_iv: "f5cc7904712f2e6f17fd708b529ee880"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.317416 16710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.322283 16710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.323827 16710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.327165 16710 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:33343
I20260809 02:14:59.327504 16891 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:33343 every 8 connection(s)
I20260809 02:14:59.327595 16710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
I20260809 02:14:59.328373 16892 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:14:59.328963 16892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Bootstrap starting.
I20260809 02:14:59.331197 16892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:14:59.331446 16892 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: Bootstrap complete.
I20260809 02:14:59.331661 16892 raft_consensus.cc:348] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } }
I20260809 02:14:59.331730 16892 raft_consensus.cc:729] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 45fbacfb7e304be99f48e7d0c9660e63, State: Initialized, Role: LEARNER
I20260809 02:14:59.331758 16892 consensus_queue.cc:260] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } }
I20260809 02:14:59.331928 16892 sys_catalog.cc:564] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:59.332005 16896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } }
I20260809 02:14:59.332110 16896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:14:59.333472 16907 catalog_manager.cc:1259] Loaded cluster ID: df6bed2e7236435094129a9bfbaea2ee
I20260809 02:14:59.333557 16907 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: loading cluster ID for follower catalog manager: success
I20260809 02:14:59.334762 16907 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: acquiring CA information for follower catalog manager: success
I20260809 02:14:59.335101 16907 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:14:59.352051 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.344711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36312 (local address 127.9.40.125:33343)
0809 02:14:59.344883 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.344885 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.344897 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:14:59.344936 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.344937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.344949 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.345031 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.345602 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.346603 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.347356 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.347660 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.347744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.349114 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.349115 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.349115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.349123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.350835 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.351192 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.351193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.351193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.351263 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.351465 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.351467 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.351467 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.351616 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.351726 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.351808 (+ 82us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.351902 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
WARNING: no policy specified for oryx/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.124@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.124.keytab.
Entry for principal oryx/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.124.keytab.
I20260809 02:14:59.380405 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46387
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.124.keytab
--principal=oryx/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:14:59.467799 16912 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:59.468024 16912 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:59.468047 16912 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:59.469273 16912 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:14:59.469321 16912 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:59.469336 16912 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:59.469379 16912 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:14:59.469408 16912 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:14:59.470957 16912 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.124.keytab
--principal=oryx/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46387
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16912
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:59.471235 16912 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:59.471496 16912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.476547 16917 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.476769 16919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.476547 16921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.476850 16912 server_base.cc:1034] running on GCE node
I20260809 02:14:59.477115 16912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.477442 16912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.478614 16912 hybrid_clock.cc:648] HybridClock initialized: now 1786241699478591 us; error 44 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.481290 16912 init.cc:377] Logged in from keytab as oryx/127.9.40.124@KRBTEST.COM (short username oryx)
I20260809 02:14:59.481804 16912 webserver.cc:460] Webserver started at http://127.9.40.124:33701/ using document root <none> and password file <none>
I20260809 02:14:59.482059 16912 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.482129 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.482283 16912 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:14:59.483392 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/instance:
uuid: "a79632bd01b4471c8ec2403909d485ec"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "0914d0c9b2ae846adaac5de35fabf68c"
server_key_iv: "95f883296acf0f0699a0f24c6db4e153"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.483780 16912 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal/instance:
uuid: "a79632bd01b4471c8ec2403909d485ec"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "0914d0c9b2ae846adaac5de35fabf68c"
server_key_iv: "95f883296acf0f0699a0f24c6db4e153"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.485317 16912 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:14:59.486249 16928 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.486459 16912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:14:59.486567 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "a79632bd01b4471c8ec2403909d485ec"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "0914d0c9b2ae846adaac5de35fabf68c"
server_key_iv: "95f883296acf0f0699a0f24c6db4e153"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.486678 16912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.498260 16912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.500085 16912 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:59.500196 16912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.504357 16912 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:46387
I20260809 02:14:59.504408 16989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:46387 every 8 connection(s)
I20260809 02:14:59.504869 16912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
I20260809 02:14:59.506031 16990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:14:59.507118 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 16912
I20260809 02:14:59.508234 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260809 02:14:59.519320 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.509453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45678 (local address 127.9.40.126:37597)
0809 02:14:59.509497 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.509499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.509694 (+ 195us) server_negotiation.cc:407] Connection header received
0809 02:14:59.509776 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.509776 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.509784 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.509833 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.510421 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.511159 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.512720 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.513117 (+ 397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.513191 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.515647 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.515649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.515649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.515659 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.517544 (+ 1885us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.518075 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.518077 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.518077 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.518158 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.518535 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.518536 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.518536 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.518682 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.518789 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.519121 (+ 332us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.519217 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:14:59.519531 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.509327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45678)
0809 02:14:59.509578 (+ 251us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.509619 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.509706 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.509872 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.509875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.510214 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.510321 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.510325 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.511280 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.511281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.512594 (+ 1313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.512597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.513223 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.513224 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.513315 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.513753 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.513764 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.515510 (+ 1746us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.517611 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.517619 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.517621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.517959 (+ 338us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.518246 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.518247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.518247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.518438 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.518787 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.518795 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.518952 (+ 157us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.519072 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.519248 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":101,"threads_started":1}
I20260809 02:14:59.519680 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.507909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49712 (local address 127.9.40.124:46387)
0809 02:14:59.508197 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.508205 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.508250 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:14:59.508825 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.508835 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.508903 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.509064 (+ 161us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.510000 (+ 936us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.511256 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.512412 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.512949 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.513067 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.516223 (+ 3156us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.516232 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.516232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.516240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.518401 (+ 2161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.518776 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.518778 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.518779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.518839 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.519052 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.519053 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.519053 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.519194 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.519336 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.519426 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.519504 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":242,"thread_start_us":172,"threads_started":1}
I20260809 02:14:59.520466 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:14:59.526836 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.520803 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36316)
0809 02:14:59.520843 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.520856 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.520900 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.521026 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.521026 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.521140 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.521188 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.521190 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.521827 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.521828 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.522317 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.522319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.522706 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.522707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.522760 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.523110 (+ 350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.523114 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.524277 (+ 1163us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.526080 (+ 1803us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.526081 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.526081 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.526229 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.526385 (+ 156us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.526385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.526385 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.526432 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.526614 (+ 182us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.526614 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.526657 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.526706 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.526774 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 02:14:59.526854 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.520861 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36316 (local address 127.9.40.125:33343)
0809 02:14:59.520919 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.520921 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.520942 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:14:59.520969 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.520970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.520983 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.521025 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.521235 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.521762 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.522399 (+ 637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.522635 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.522696 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.524365 (+ 1669us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.524366 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.524366 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.524374 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.526007 (+ 1633us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.526279 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.526280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.526280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.526328 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.526470 (+ 142us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.526471 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.526471 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.526564 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.526619 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.526724 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.526788 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:59.527274 16990 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260809 02:14:59.533726 16993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.527546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49720)
0809 02:14:59.527575 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.527586 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.527641 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.527745 (+ 104us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.527746 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.527849 (+ 103us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.527876 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.527876 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.528537 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.528538 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.529101 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.529103 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.529443 (+ 340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.529444 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.529480 (+ 36us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.529824 (+ 344us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.529828 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.530911 (+ 1083us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.532693 (+ 1782us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.532693 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.532694 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.532951 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.533157 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.533158 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.533158 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.533210 (+ 52us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.533398 (+ 188us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.533399 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.533430 (+ 31us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.533479 (+ 49us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.533643 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":15}
I20260809 02:14:59.533820 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.527577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49720 (local address 127.9.40.124:46387)
0809 02:14:59.527640 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.527642 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.527652 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:59.527675 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.527675 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.527683 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.527741 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.527936 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.528459 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.529174 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.529376 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.529435 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.530989 (+ 1554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.530990 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.530990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.530997 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.532621 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.533037 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.533038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.533038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.533091 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.533251 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.533251 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.533252 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.533345 (+ 93us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.533393 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.533529 (+ 136us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.533681 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":43}
I20260809 02:14:59.535125 16990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Bootstrap starting.
I20260809 02:14:59.535974 16990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Neither blocks nor log segments found. Creating new log.
I20260809 02:14:59.536284 16990 log.cc:824] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Log is configured to *not* fsync() on all Append() calls
I20260809 02:14:59.537106 16990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: No bootstrap required, opened a new log
I20260809 02:14:59.538656 16990 raft_consensus.cc:348] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } }
I20260809 02:14:59.538790 16990 raft_consensus.cc:374] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:14:59.538813 16990 raft_consensus.cc:729] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a79632bd01b4471c8ec2403909d485ec, State: Initialized, Role: FOLLOWER
I20260809 02:14:59.538957 16990 consensus_queue.cc:260] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } }
I20260809 02:14:59.539321 16990 sys_catalog.cc:564] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:59.539274 16995 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } } }
I20260809 02:14:59.539531 16995 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:14:59.541131 17006 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:14:59.541221 17006 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:14:59.543782 16912 master_runner.cc:428] Detected that this master a79632bd01b4471c8ec2403909d485ec is joining an existing cluster
I20260809 02:14:59.543855 16912 master_runner.cc:432] Initiating AddMaster RPC to add 127.9.40.124:46387
I20260809 02:14:59.552626 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.545330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36320 (local address 127.9.40.125:33343)
0809 02:14:59.545376 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.545378 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.545520 (+ 142us) server_negotiation.cc:407] Connection header received
0809 02:14:59.545614 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.545615 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.545626 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.545667 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.546098 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.546877 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548143 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548504 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548570 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.549835 (+ 1265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.549836 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.549837 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.549844 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.551340 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.551754 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.551755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.551756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.551815 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.552054 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.552055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.552055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.552182 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.552269 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.552438 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.552526 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:14:59.552649 17016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.545343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36320)
0809 02:14:59.545465 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.545475 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.545541 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.545739 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.545740 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.545948 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.546007 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.546009 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.546985 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.546985 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548037 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.548040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548644 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548761 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.549364 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.549369 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.549703 (+ 334us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.551452 (+ 1749us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.551453 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.551454 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.551655 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.551902 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.551903 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.551903 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.551979 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.552260 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.552261 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.552310 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.552410 (+ 100us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.552523 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":106,"mutex_wait_us":33,"thread_start_us":81,"threads_started":1}
I20260809 02:14:59.571213 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16262
I20260809 02:14:59.578791 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.545574 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49734 (local address 127.9.40.124:46387)
0809 02:14:59.545614 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.545616 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.546181 (+ 565us) server_negotiation.cc:407] Connection header received
0809 02:14:59.546261 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.546262 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.546269 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.546293 (+ 24us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.546698 (+ 405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.547387 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548367 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548646 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548685 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.575518 (+ 26833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.575519 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.575520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.575528 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.577547 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.577952 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.577953 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.577953 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.578007 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.578178 (+ 171us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.578178 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.578179 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.578320 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.578360 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.578556 (+ 196us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.578644 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:14:59.579188 17017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.545653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49734)
0809 02:14:59.546131 (+ 478us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.546141 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.546200 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.546367 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.546368 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.546542 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.546603 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.546604 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.547476 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.547476 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548275 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.548277 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548787 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548847 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.574664 (+ 25817us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.574671 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.575377 (+ 706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.577644 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.577645 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.577645 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.577868 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.578067 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.578068 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.578068 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.578121 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.578397 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.578398 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.578447 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.578927 (+ 480us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.579039 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"thread_start_us":71,"threads_started":1}
I20260809 02:14:59.580475 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:40255
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:37597
--webserver_port=40047
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:14:59.581629 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.545048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45690 (local address 127.9.40.126:37597)
0809 02:14:59.545105 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.545107 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.545218 (+ 111us) server_negotiation.cc:407] Connection header received
0809 02:14:59.545274 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.545275 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.545289 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.545352 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.545891 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.546757 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548154 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548502 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548552 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.577767 (+ 29215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.577769 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.577769 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.577779 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.579953 (+ 2174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.580610 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.580612 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.580612 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.580708 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.580962 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.580963 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.580964 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.581121 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.581170 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.581415 (+ 245us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.581524 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:14:59.582010 17015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.544942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45690)
0809 02:14:59.545131 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.545143 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.545211 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.545425 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.545427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.545679 (+ 252us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.545785 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.545787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.546855 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.546856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548037 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.548040 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.548642 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.548643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.548752 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.577024 (+ 28272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.577030 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.577618 (+ 588us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.580200 (+ 2582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.580201 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.580202 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.580502 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.580817 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.580818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.580818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.580882 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.581222 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.581223 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.581287 (+ 64us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.581766 (+ 479us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.581890 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"mutex_wait_us":47,"thread_start_us":115,"threads_started":1}
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241698, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
I20260809 02:14:59.593552 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.584573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49744)
0809 02:14:59.584620 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.584633 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.584682 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.584968 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.584969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.585173 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.585227 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.585229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.586313 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.586314 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.587230 (+ 916us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.587232 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.587836 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.587837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.587909 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.588401 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.588407 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.590283 (+ 1876us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.592323 (+ 2040us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.592324 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.592325 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.592566 (+ 241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.592828 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.592829 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.592829 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.592893 (+ 64us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.593201 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.593202 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.593248 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.593317 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.593438 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":23}
I20260809 02:14:59.593956 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.584646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49744 (local address 127.9.40.124:46387)
0809 02:14:59.584696 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.584699 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.584718 (+ 19us) server_negotiation.cc:407] Connection header received
0809 02:14:59.584806 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.584807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.584820 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.584894 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.585335 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.586195 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.587333 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.587714 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.587779 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.590388 (+ 2609us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.590390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.590390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.590397 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.592221 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.592658 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.592659 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.592660 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.592718 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.592965 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.592966 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.592967 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.593106 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.593172 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.593766 (+ 594us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.593851 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:14:59.594488 16188 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.9.40.124:46387: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "a79632bd01b4471c8ec2403909d485ec"
member_type: NON_VOTER
last_known_addr {
host: "127.9.40.124"
port: 46387
}
attrs {
promote: true
}
}
dest_uuid: "dd39532293ad4e55ae39a0f51b8d1606"
cas_config_opid_index: 5
I20260809 02:14:59.594754 16188 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } }
I20260809 02:14:59.595155 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a79632bd01b4471c8ec2403909d485ec (127.9.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.595611 16959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:14:59.596138 16811 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.9.40.124:46387
I20260809 02:14:59.596210 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.596269 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:59.597203 16959 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:14:59.597512 16912 master.cc:537] Master@127.9.40.124:46387 shutting down...
I20260809 02:14:59.597501 16812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:14:59.599252 16173 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 -> Peer a79632bd01b4471c8ec2403909d485ec (127.9.40.124:46387): Couldn't send request to peer a79632bd01b4471c8ec2403909d485ec. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:14:59.601258 16912 raft_consensus.cc:2229] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:14:59.601351 16912 raft_consensus.cc:2258] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:14:59.601393 16912 tablet_replica.cc:330] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: stopping tablet replica
I20260809 02:14:59.603569 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.595306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36322)
0809 02:14:59.599339 (+ 4033us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.599351 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.599407 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.599851 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.599852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.599854 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:14:59.599898 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.599900 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.600957 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.600958 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.602306 (+ 1348us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.602310 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.603277 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.603278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.603353 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.603377 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.603446 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.603490 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4015}
I20260809 02:14:59.603783 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.596441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36322 (local address 127.9.40.125:33343)
0809 02:14:59.596786 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.596788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.599485 (+ 2697us) server_negotiation.cc:407] Connection header received
0809 02:14:59.599521 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.599523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.599550 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.599615 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:14:59.599989 (+ 374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.600847 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.602418 (+ 1571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.603089 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.603168 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.603238 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.603674 (+ 436us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.603715 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267}
I20260809 02:14:59.604082 16865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 LEARNER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260809 02:14:59.604295 16812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260809 02:14:59.604931 16865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER a79632bd01b4471c8ec2403909d485ec (127.9.40.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.606320 16812 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: attempting to promote NON_VOTER 45fbacfb7e304be99f48e7d0c9660e63 to VOTER
I20260809 02:14:59.606388 16896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd39532293ad4e55ae39a0f51b8d1606. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.606484 16896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:14:59.606521 16896 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: NON_VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: true } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.606515 16812 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } }
I20260809 02:14:59.606561 16896 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: LEARNER
I20260809 02:14:59.607131 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.606975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49752)
0809 02:14:59.607045 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.607101 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:14:59.607127 16865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 LEARNER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260809 02:14:59.607414 17021 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260809 02:14:59.607698 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.607516 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49756)
0809 02:14:59.607642 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.607670 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
W20260809 02:14:59.607828 16173 consensus_peers.cc:487] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 -> Peer a79632bd01b4471c8ec2403909d485ec (127.9.40.124:46387): Couldn't send request to peer a79632bd01b4471c8ec2403909d485ec. Status: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46387: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:14:59.608016 16811 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.608155 16865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.609093 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.609174 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:59.609278 16811 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.609323 16811 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:14:59.609535 17020 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.609675 17020 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:14:59.614672 16912 master.cc:552] Master@127.9.40.124:46387 shutdown complete.
I20260809 02:14:59.614725 16912 master_runner.cc:305] Clearing existing system tablet
I20260809 02:14:59.614861 16912 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.614900 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.615669 16912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.616231 17025 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.616375 16912 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.616425 16912 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.616447 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "a79632bd01b4471c8ec2403909d485ec"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "0914d0c9b2ae846adaac5de35fabf68c"
server_key_iv: "95f883296acf0f0699a0f24c6db4e153"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.617475 16912 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:14:59.619122 16912 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260809 02:14:59.619199 16912 log.cc:1196] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260809 02:14:59.619395 16912 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Deleting consensus metadata
I20260809 02:14:59.619640 16912 master_runner.cc:315] Copying system tablet from 127.9.40.126:37597
I20260809 02:14:59.620296 16912 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.126:37597
I20260809 02:14:59.627995 17031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.620664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:45692)
0809 02:14:59.620877 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.620888 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.620965 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.621127 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.621127 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.621340 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.621395 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.621397 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.622439 (+ 1042us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.622440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.623296 (+ 856us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.623299 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.623797 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.623798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.623863 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.624407 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.624413 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.624896 (+ 483us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.626791 (+ 1895us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.626792 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.626793 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.627026 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.627267 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.627268 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.627268 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.627333 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.627649 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.627650 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.627700 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.627773 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.627883 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":154,"threads_started":1}
I20260809 02:14:59.628013 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.620770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45692 (local address 127.9.40.126:37597)
0809 02:14:59.620817 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.620819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.620976 (+ 157us) server_negotiation.cc:407] Connection header received
0809 02:14:59.621014 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.621015 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.621027 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.621107 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.621492 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.622335 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.623404 (+ 1069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.623710 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.623785 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.624999 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.625000 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.625001 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.625008 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.626693 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.627118 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.627119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.627119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.627186 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.627417 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.627418 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.627418 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.627570 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.627648 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.627830 (+ 182us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.627918 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:59.628448 16211 tablet_copy_service.cc:140] P dd39532293ad4e55ae39a0f51b8d1606: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer a79632bd01b4471c8ec2403909d485ec ({username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:45692)
I20260809 02:14:59.628517 16211 tablet_copy_service.cc:161] P dd39532293ad4e55ae39a0f51b8d1606: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer a79632bd01b4471c8ec2403909d485ec at {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:45692: session id = a79632bd01b4471c8ec2403909d485ec-00000000000000000000000000000000
I20260809 02:14:59.629177 16211 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:14:59.629648 16912 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260809 02:14:59.631044 16912 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: tablet copy: Starting download of 0 data blocks...
I20260809 02:14:59.631189 16912 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: tablet copy: Starting download of 1 WAL segments...
I20260809 02:14:59.632392 16912 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:14:59.633503 16211 tablet_copy_service.cc:342] P dd39532293ad4e55ae39a0f51b8d1606: Request end of tablet copy session a79632bd01b4471c8ec2403909d485ec-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:45692
I20260809 02:14:59.633567 16211 tablet_copy_service.cc:433] P dd39532293ad4e55ae39a0f51b8d1606: ending tablet copy session a79632bd01b4471c8ec2403909d485ec-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer a79632bd01b4471c8ec2403909d485ec
I20260809 02:14:59.637082 16912 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.638329 17032 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.638338 17036 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.638662 17033 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.639328 16912 server_base.cc:1034] running on GCE node
I20260809 02:14:59.639448 16912 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.639598 16912 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.640828 16912 hybrid_clock.cc:648] HybridClock initialized: now 1786241699640819 us; error 26 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.642561 16912 init.cc:377] Logged in from keytab as oryx/127.9.40.124@KRBTEST.COM (short username oryx)
I20260809 02:14:59.642877 16912 webserver.cc:460] Webserver started at http://127.9.40.124:35691/ using document root <none> and password file <none>
I20260809 02:14:59.643028 16912 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.643069 16912 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.643880 16912 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.644644 17043 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.644860 16912 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.644909 16912 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "a79632bd01b4471c8ec2403909d485ec"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "0914d0c9b2ae846adaac5de35fabf68c"
server_key_iv: "95f883296acf0f0699a0f24c6db4e153"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.644968 16912 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.669364 16912 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.671475 16912 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.674737 16912 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:46387
I20260809 02:14:59.675028 17102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:46387 every 8 connection(s)
I20260809 02:14:59.675123 16912 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
I20260809 02:14:59.676124 17103 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:14:59.676689 17103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Bootstrap starting.
I20260809 02:14:59.680366 17103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:14:59.680629 17103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Bootstrap complete.
I20260809 02:14:59.680903 17103 raft_consensus.cc:348] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } }
I20260809 02:14:59.681033 17103 raft_consensus.cc:729] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: a79632bd01b4471c8ec2403909d485ec, State: Initialized, Role: LEARNER
I20260809 02:14:59.681077 17103 consensus_queue.cc:260] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } }
I20260809 02:14:59.681282 17103 sys_catalog.cc:564] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:14:59.681303 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:14:59.681576 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: LEARNER
I20260809 02:14:59.683043 17118 catalog_manager.cc:1259] Loaded cluster ID: df6bed2e7236435094129a9bfbaea2ee
I20260809 02:14:59.683598 17118 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: loading cluster ID for follower catalog manager: success
I20260809 02:14:59.684857 17118 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: acquiring CA information for follower catalog manager: success
I20260809 02:14:59.685241 17118 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:14:59.692312 17018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:59.692530 17018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:59.692559 17018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:59.693953 17018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:59.694003 17018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:59.694034 17018 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:14:59.695750 17018 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:40255
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=40047
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:59.696044 17018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:59.696302 17018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.702230 17123 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.702358 17126 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.702338 17124 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.702595 17018 server_base.cc:1034] running on GCE node
I20260809 02:14:59.702845 17018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.703176 17018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.704370 17018 hybrid_clock.cc:648] HybridClock initialized: now 1786241699704355 us; error 29 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.707362 17018 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260809 02:14:59.707815 17018 webserver.cc:460] Webserver started at http://127.9.40.65:40047/ using document root <none> and password file <none>
I20260809 02:14:59.708058 17018 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.708146 17018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.709709 17018 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:14:59.710423 17136 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.710649 17018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.710769 17018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "d5d452ec7a4940c39cea6b49ab41a406"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "37c26a555ab7804a21c6ea01bf5f7417"
server_key_iv: "473269232d820dad8a3bb888de41f085"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.711134 17018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.725162 17018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.727329 17018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:59.727509 17018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.727774 17018 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:59.728106 17018 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:59.728173 17018 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.728221 17018 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:59.728251 17018 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.734694 17018 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:40255
I20260809 02:14:59.734779 17261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:40255 every 8 connection(s)
I20260809 02:14:59.735157 17018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:14:59.736637 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17018
I20260809 02:14:59.736830 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16409
I20260809 02:14:59.752061 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:35527
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:37597
--webserver_port=43343
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:14:59.759738 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.735957 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:32817 (local address 127.9.40.126:37597)
0809 02:14:59.736037 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.736039 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.736297 (+ 258us) server_negotiation.cc:407] Connection header received
0809 02:14:59.747881 (+ 11584us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.747884 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.747900 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.747934 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.749436 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750254 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752045 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752354 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752397 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.755466 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.755468 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.755468 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.755478 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.757923 (+ 2445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758546 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.758547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.758547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.758607 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758934 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.758935 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.758936 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.759076 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.759136 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.759538 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.759636 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:14:59.759738 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.736641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:49683 (local address 127.9.40.125:33343)
0809 02:14:59.736694 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.736696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.736704 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:14:59.747891 (+ 11187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.747893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.747908 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.747940 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.749070 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750254 (+ 1184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752045 (+ 1791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752354 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752397 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.756409 (+ 4012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.756410 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.756410 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.756425 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.757897 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758546 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.758547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.758547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.758607 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758945 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.758947 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.758947 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.759097 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.759147 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.759538 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.759635 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:14:59.760325 17266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.735944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:32817)
0809 02:14:59.736127 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.736166 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.747706 (+ 11540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.748100 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.748105 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.749221 (+ 1116us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.749289 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.749290 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750485 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.750485 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.751933 (+ 1448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.751936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752500 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.752647 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.753483 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.753494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.755344 (+ 1850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.758029 (+ 2685us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758031 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.758031 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.758446 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.758682 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.758683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.758848 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.759175 (+ 327us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.759182 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.759347 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.759846 (+ 499us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.760069 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":99,"thread_start_us":97,"threads_started":1}
I20260809 02:14:59.760342 17269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.736472 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.9.40.65:49683)
0809 02:14:59.736656 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.736665 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.747724 (+ 11059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.748101 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.748105 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.748646 (+ 541us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.748957 (+ 311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.748963 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750470 (+ 1507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.750473 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.751933 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.751936 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752500 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752500 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.752647 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.753483 (+ 836us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.753494 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.756319 (+ 2825us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.758006 (+ 1687us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758014 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.758016 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.758446 (+ 430us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.758682 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.758683 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.758683 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.758848 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.759181 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.759184 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.759347 (+ 163us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.759912 (+ 565us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.760119 (+ 207us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":176,"thread_start_us":67,"threads_started":1}
I20260809 02:14:59.761193 17263 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:14:59.761231 17264 heartbeater.cc:347] Connected to a master server at 127.9.40.125:33343
I20260809 02:14:59.761338 17263 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.761337 17264 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.761588 17264 heartbeater.cc:511] Master 127.9.40.125:33343 requested a full tablet report, sending...
I20260809 02:14:59.761591 17263 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:14:59.762262 17267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.736237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.9.40.65:60585)
0809 02:14:59.736382 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.736393 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.747724 (+ 11331us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.748166 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.748167 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.748641 (+ 474us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.748958 (+ 317us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.748964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750606 (+ 1642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.750607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.751907 (+ 1300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.751910 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752585 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.752659 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.753516 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.753522 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.755838 (+ 2316us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.759054 (+ 3216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.759055 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.759055 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.759330 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.759621 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.759622 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.759622 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.759698 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.761846 (+ 2148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.761847 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.761913 (+ 66us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.761976 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.762108 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"mutex_wait_us":40,"thread_start_us":89,"threads_started":1}
I20260809 02:14:59.762326 16188 ts_manager.cc:194] Re-registered known tserver with Master: d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
I20260809 02:14:59.762449 16855 ts_manager.cc:194] Registered new tserver with Master: d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
I20260809 02:14:59.762717 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.736344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60585 (local address 127.9.40.124:46387)
0809 02:14:59.736514 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.736516 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.736526 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:14:59.748044 (+ 11518us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.748045 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.748061 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.748207 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.749405 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.750505 (+ 1100us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752007 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.752489 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.752921 (+ 432us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.755938 (+ 3017us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.755939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.755940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.755949 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.758932 (+ 2983us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.759455 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.759457 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.759457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.759529 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.760314 (+ 785us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.760316 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.760316 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.761729 (+ 1413us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.762459 (+ 730us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.762510 (+ 51us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.762612 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":70,"threads_started":1}
I20260809 02:14:59.762892 16188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:32817
I20260809 02:14:59.763092 17262 heartbeater.cc:347] Connected to a master server at 127.9.40.124:46387
I20260809 02:14:59.763157 17262 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.763239 17262 heartbeater.cc:511] Master 127.9.40.124:46387 requested a full tablet report, sending...
I20260809 02:14:59.763863 17063 ts_manager.cc:194] Registered new tserver with Master: d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
W20260809 02:14:59.847348 17270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:59.847549 17270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:59.847573 17270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:59.848798 17270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:59.848839 17270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:59.848886 17270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:14:59.850407 17270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:35527
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=43343
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:59.850662 17270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:59.850903 17270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.856178 17278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.856230 17270 server_base.cc:1034] running on GCE node
W20260809 02:14:59.856344 17275 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.856447 17276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.856709 17270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.857033 17270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.858225 17270 hybrid_clock.cc:648] HybridClock initialized: now 1786241699858221 us; error 37 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:14:59.861019 17270 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260809 02:14:59.861486 17270 webserver.cc:460] Webserver started at http://127.9.40.66:43343/ using document root <none> and password file <none>
I20260809 02:14:59.861776 17270 fs_manager.cc:359] Metadata directory not provided
I20260809 02:14:59.861868 17270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:14:59.863512 17270 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.864171 17285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.864383 17270 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:14:59.864508 17270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "fa991920d9a247419c0a95b7afeca1ca"
format_stamp: "Formatted at 2026-08-09 02:14:58 on dist-test-slave-qg60"
server_key: "914d11e5f69940f08054e80ce55e58d9"
server_key_iv: "1c88a23cd148270dadcec764a6a74e04"
server_key_version: "encryptionkey@0"
I20260809 02:14:59.864907 17270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:14:59.879753 17270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:14:59.882400 17270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:14:59.882576 17270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:14:59.882822 17270 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:14:59.883198 17270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:14:59.883234 17270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.883256 17270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:14:59.883270 17270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:14:59.889571 17270 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35527
I20260809 02:14:59.889624 17408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35527 every 8 connection(s)
I20260809 02:14:59.889974 17270 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:14:59.891467 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17270
I20260809 02:14:59.891631 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16559
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:14:59.900611 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:35429
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:37597
--webserver_port=43645
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41241
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:14:59.903724 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.891076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:40633 (local address 127.9.40.124:46387)
0809 02:14:59.891124 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.891126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.891153 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:14:59.891942 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.891943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.891956 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.891987 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.893569 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.894337 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.895537 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.895917 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.895989 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.899724 (+ 3735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.899726 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.899726 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.899736 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.901714 (+ 1978us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.902507 (+ 793us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.902508 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.902509 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.902581 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.902880 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.902881 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.902882 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.903027 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.903072 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.903483 (+ 411us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.903598 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:14:59.903731 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.894034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35449 (local address 127.9.40.125:33343)
0809 02:14:59.894108 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.894110 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.894127 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:14:59.894869 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.894871 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.894885 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.894943 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.895373 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.895904 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.896615 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.896947 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.897009 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.899710 (+ 2701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.899711 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.899712 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.899720 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.901837 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.902475 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.902476 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.902477 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.902537 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.903085 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.903086 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.903086 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.903249 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.903294 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.903528 (+ 234us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.903618 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:14:59.904356 17415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.891177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.9.40.66:35449)
0809 02:14:59.891317 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.891327 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.894743 (+ 3416us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.894968 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.894969 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.895198 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.895284 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.895285 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.895999 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.896000 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.896528 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.896530 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.897065 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.897066 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.897149 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.897572 (+ 423us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.897577 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.899602 (+ 2025us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.901937 (+ 2335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.901938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.901939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.902388 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.902916 (+ 528us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.902917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.902917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.902988 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.903350 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.903351 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.903398 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.903873 (+ 475us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.904062 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":74,"threads_started":1}
I20260809 02:14:59.904861 17411 heartbeater.cc:347] Connected to a master server at 127.9.40.125:33343
I20260809 02:14:59.904968 17411 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.905176 17411 heartbeater.cc:511] Master 127.9.40.125:33343 requested a full tablet report, sending...
I20260809 02:14:59.905481 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.890731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52099 (local address 127.9.40.126:37597)
0809 02:14:59.890798 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:14:59.890800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:14:59.891159 (+ 359us) server_negotiation.cc:407] Connection header received
0809 02:14:59.897423 (+ 6264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:14:59.897425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:14:59.897438 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:14:59.897490 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:14:59.897956 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.898652 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.899384 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.899707 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.899752 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.902385 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:14:59.902386 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:14:59.902387 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:14:59.902395 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:14:59.904121 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.904548 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.904549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.904550 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.904650 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:14:59.904911 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:14:59.904912 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:14:59.904912 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:14:59.905045 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:14:59.905089 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 02:14:59.905303 (+ 214us) server_negotiation.cc:299] Negotiation successful
0809 02:14:59.905381 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:14:59.905624 16855 ts_manager.cc:194] Registered new tserver with Master: fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527)
I20260809 02:14:59.906369 17413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.890724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:52099)
0809 02:14:59.890906 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.891062 (+ 156us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.897323 (+ 6261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.897494 (+ 171us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.897495 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.897782 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.897851 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.897853 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.898736 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.898737 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.899281 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.899283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.899843 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.899844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.899917 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.900589 (+ 672us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.900596 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.902272 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.904214 (+ 1942us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.904214 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.904215 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.904453 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.904760 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.904761 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.904761 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.904830 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.905139 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.905140 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.905180 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.905564 (+ 384us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.906245 (+ 681us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":578,"thread_start_us":93,"threads_started":1}
I20260809 02:14:59.906370 17414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:14:59.890977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.9.40.66:40633)
0809 02:14:59.891095 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:14:59.891105 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:14:59.891814 (+ 709us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:14:59.892158 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:14:59.892162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:14:59.893078 (+ 916us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:14:59.893437 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.893445 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.894436 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.894437 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.895448 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:14:59.895451 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:14:59.896059 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:14:59.896060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:14:59.896217 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:14:59.896947 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:14:59.896957 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:14:59.899616 (+ 2659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:14:59.901814 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.901822 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.901825 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.902390 (+ 565us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.902661 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:14:59.902662 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:14:59.902662 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:14:59.902803 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:14:59.903122 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:14:59.903129 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:14:59.903303 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:14:59.904302 (+ 999us) client_negotiation.cc:240] Negotiation successful
0809 02:14:59.906161 (+ 1859us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260809 02:14:59.906801 17409 heartbeater.cc:347] Connected to a master server at 127.9.40.124:46387
I20260809 02:14:59.906865 17409 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.906944 17409 heartbeater.cc:511] Master 127.9.40.124:46387 requested a full tablet report, sending...
I20260809 02:14:59.907060 17410 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:14:59.907143 17410 heartbeater.cc:464] Registering TS with master...
I20260809 02:14:59.907218 17410 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:14:59.907269 17063 ts_manager.cc:194] Registered new tserver with Master: fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527)
I20260809 02:14:59.907466 16188 ts_manager.cc:194] Re-registered known tserver with Master: fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527)
I20260809 02:14:59.908018 16188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:52099
W20260809 02:14:59.988119 17416 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:14:59.988337 17416 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:14:59.988364 17416 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:14:59.989676 17416 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:14:59.989732 17416 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:14:59.989761 17416 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:14:59.991451 17416 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41241
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:35429
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=43645
--tserver_master_addrs=127.9.40.126:37597,127.9.40.125:33343,127.9.40.124:46387
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17416
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:14:59.991721 17416 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:14:59.991986 17416 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:14:59.997457 17422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:14:59.997434 17424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.997478 17416 server_base.cc:1034] running on GCE node
W20260809 02:14:59.997434 17421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:14:59.997864 17416 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:14:59.998126 17416 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:14:59.999303 17416 hybrid_clock.cc:648] HybridClock initialized: now 1786241699999299 us; error 39 us; skew 500 ppm
Aug 09 02:14:59 dist-test-slave-qg60 krb5kdc[16141](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:00.002074 17416 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260809 02:15:00.002511 17416 webserver.cc:460] Webserver started at http://127.9.40.67:43645/ using document root <none> and password file <none>
I20260809 02:15:00.002701 17416 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:00.002760 17416 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:00.004128 17416 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:00.004801 17431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:00.004995 17416 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:00.005088 17416 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "8ddcde33c86445df9da586dc95035fae"
format_stamp: "Formatted at 2026-08-09 02:14:59 on dist-test-slave-qg60"
server_key: "5b1add006d5cb056f76d9763c050a10e"
server_key_iv: "69147d0829888361255f2822452e6030"
server_key_version: "encryptionkey@0"
I20260809 02:15:00.005460 17416 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:00.017613 17416 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:00.019817 17416 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:00.019994 17416 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:00.020249 17416 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:00.020579 17416 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:00.020642 17416 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:00.020689 17416 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:00.020713 17416 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:00.026623 17416 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35429
I20260809 02:15:00.026713 17558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35429 every 8 connection(s)
I20260809 02:15:00.027127 17416 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:15:00 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
Aug 09 02:15:00 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.124@KRBTEST.COM
Aug 09 02:15:00 dist-test-slave-qg60 krb5kdc[16141](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241699, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.125@KRBTEST.COM
I20260809 02:15:00.037338 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17416
I20260809 02:15:00.039570 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.028557 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57083 (local address 127.9.40.126:37597)
0809 02:15:00.028616 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.028618 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.028628 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:00.029360 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.029362 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.029374 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.029430 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.030183 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.030702 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.032170 (+ 1468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.032356 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.032411 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.036148 (+ 3737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.036149 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.036149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.036157 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.037858 (+ 1701us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038442 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.038444 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.038444 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.038509 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038824 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.038825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.038825 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.038953 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.039001 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.039386 (+ 385us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.039477 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:15:00.039752 17566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.028252 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.9.40.67:56827)
0809 02:15:00.028408 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.028420 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.028978 (+ 558us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.029314 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.029315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.029807 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:00.030060 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.030066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.032137 (+ 2071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.032138 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.032872 (+ 734us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.032875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.033400 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.033401 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.033466 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.033840 (+ 374us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:00.033845 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:00.035575 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:00.038197 (+ 2622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038198 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.038199 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.038463 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.038677 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038678 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.038678 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.038741 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.039043 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:00.039050 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:00.039224 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.039337 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.039502 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"mutex_wait_us":12,"thread_start_us":76,"threads_started":1}
I20260809 02:15:00.039894 17567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.028430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:57083)
0809 02:15:00.028545 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.028554 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.028974 (+ 420us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.029444 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.029445 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.029807 (+ 362us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:00.030060 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.030066 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.030812 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.030813 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.032083 (+ 1270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.032085 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.032441 (+ 356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.032442 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.032623 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.033262 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:00.033273 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:00.036030 (+ 2757us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:00.037963 (+ 1933us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.037972 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.037975 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.038341 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.038598 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038599 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.038599 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.038741 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.039050 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:00.039050 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:00.039224 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.039691 (+ 467us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.039794 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":99,"mutex_wait_us":31,"thread_start_us":68,"threads_started":1}
I20260809 02:15:00.040280 17560 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37597
I20260809 02:15:00.040391 17560 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:00.040534 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.028333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56827 (local address 127.9.40.124:46387)
0809 02:15:00.028390 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.028392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.028490 (+ 98us) server_negotiation.cc:407] Connection header received
0809 02:15:00.029229 (+ 739us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.029231 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.029243 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.029292 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.030217 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.030944 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.032945 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.033269 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.033342 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.035686 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.035687 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.035687 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.035711 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.038074 (+ 2363us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038547 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.038548 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.038549 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.038607 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.038814 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.038815 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.038815 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.038953 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.039015 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.040314 (+ 1299us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.040402 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:15:00.040617 17560 heartbeater.cc:511] Master 127.9.40.126:37597 requested a full tablet report, sending...
I20260809 02:15:00.041023 17559 heartbeater.cc:347] Connected to a master server at 127.9.40.124:46387
I20260809 02:15:00.041077 17559 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:00.041051 16188 ts_manager.cc:194] Re-registered known tserver with Master: 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:15:00.041147 17559 heartbeater.cc:511] Master 127.9.40.124:46387 requested a full tablet report, sending...
I20260809 02:15:00.041456 17063 ts_manager.cc:194] Registered new tserver with Master: 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:15:00.041538 16188 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:57083
I20260809 02:15:00.043048 16908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.027952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:44209 (local address 127.9.40.125:33343)
0809 02:15:00.028028 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.028030 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.028303 (+ 273us) server_negotiation.cc:407] Connection header received
0809 02:15:00.029070 (+ 767us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.029072 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.029084 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.029165 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.030164 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.030931 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.033005 (+ 2074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.033322 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.033406 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.036438 (+ 3032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.036439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.036439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.036447 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.040244 (+ 3797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.042147 (+ 1903us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.042148 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.042149 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.042218 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.042457 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.042459 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.042459 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.042599 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.042664 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.042873 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.042956 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:15:00.043395 17564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.027934 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.9.40.67:44209)
0809 02:15:00.028119 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.028165 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.028959 (+ 794us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.029242 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.029247 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.029893 (+ 646us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:00.030059 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.030064 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.031022 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.031023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.032031 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.032033 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.033406 (+ 1373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.033407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.033490 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.034013 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:00.034018 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:00.036030 (+ 2012us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:00.040918 (+ 4888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.040919 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.040919 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.042039 (+ 1120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.042298 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:00.042299 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:00.042299 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:00.042359 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:00.042695 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:00.042696 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:00.042737 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.043144 (+ 407us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.043275 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":2,"thread_start_us":93,"threads_started":1}
I20260809 02:15:00.044020 17561 heartbeater.cc:347] Connected to a master server at 127.9.40.125:33343
I20260809 02:15:00.044081 17561 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:00.044159 17561 heartbeater.cc:511] Master 127.9.40.125:33343 requested a full tablet report, sending...
I20260809 02:15:00.044447 16855 ts_manager.cc:194] Registered new tserver with Master: 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:15:00.048069 16245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.038510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45694 (local address 127.9.40.126:37597)
0809 02:15:00.039671 (+ 1161us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.039673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.039873 (+ 200us) server_negotiation.cc:407] Connection header received
0809 02:15:00.039994 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.039996 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.040009 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.040061 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.041095 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.041824 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.042873 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.043155 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.043199 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.045073 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.045075 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.045075 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.045083 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.046859 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.047312 (+ 453us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.047313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.047313 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.047375 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.047563 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.047564 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.047564 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.047680 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.047784 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.047891 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.047969 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1146,"thread_start_us":110,"threads_started":1}
I20260809 02:15:00.050271 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.039491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49760 (local address 127.9.40.124:46387)
0809 02:15:00.040608 (+ 1117us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.040609 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.040618 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:00.040639 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.040640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.040656 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.040708 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.043078 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.043818 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.045230 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.045532 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.045599 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.047078 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.047079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.047080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.047087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.048861 (+ 1774us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.049270 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.049271 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.049271 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.049518 (+ 247us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.049738 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.049739 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.049740 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.049869 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.049970 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.050104 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.050182 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1102,"thread_start_us":55,"threads_started":1}
I20260809 02:15:00.051025 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.039130 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36338 (local address 127.9.40.125:33343)
0809 02:15:00.041290 (+ 2160us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.041292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.041300 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:00.041322 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.041323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.041335 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.041396 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.042273 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.043071 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.045681 (+ 2610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.046039 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.046110 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.047376 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.047377 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.047377 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.047385 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.049605 (+ 2220us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.050015 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.050016 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.050017 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.050081 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.050284 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.050285 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.050286 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.050675 (+ 389us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.050782 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.050895 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.050961 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2126,"thread_start_us":65,"threads_started":1}
I20260809 02:15:00.049933 16188 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45694:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:00.052083 16188 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:00.060142 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.056119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41090 (local address 127.9.40.65:40255)
0809 02:15:00.056901 (+ 782us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.056905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.056915 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:00.056952 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.056954 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.056969 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.057052 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.057211 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.058165 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059219 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.059763 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059857 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.059945 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.060014 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.060057 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":743,"thread_start_us":82,"threads_started":1}
I20260809 02:15:00.060171 17582 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.055856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40255 (local address 127.0.0.1:41090)
0809 02:15:00.056039 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.056052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.056094 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.057075 (+ 981us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.057076 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.057078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.057138 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.057140 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.058258 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.058259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.059128 (+ 869us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.059131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.059857 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.059929 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.059934 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.059999 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.060041 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260809 02:15:00.060420 16810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.055903 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35527 (local address 127.0.0.1:41078)
0809 02:15:00.056039 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.056052 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.056094 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.056441 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.056442 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.056444 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.056491 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.056493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.057882 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.057883 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.059018 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.059021 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.059958 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.060021 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.060025 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.060058 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.060102 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"mutex_wait_us":25,"thread_start_us":92,"threads_started":1}
I20260809 02:15:00.060768 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.055953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35429 (local address 127.0.0.1:58136)
0809 02:15:00.056502 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.056512 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.056560 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.056778 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.056779 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.056781 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.056822 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.056824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.057823 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.057824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.059227 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.059230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.060185 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.060186 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.060442 (+ 256us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.060448 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.060612 (+ 164us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.060670 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":532,"mutex_wait_us":37,"thread_start_us":86,"threads_started":1}
I20260809 02:15:00.061051 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.056218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58136 (local address 127.9.40.67:35429)
0809 02:15:00.056387 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.056391 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.056618 (+ 227us) server_negotiation.cc:407] Connection header received
0809 02:15:00.056661 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.056663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.056678 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.056777 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.056889 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.057741 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059300 (+ 1559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.060012 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.060076 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.060148 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.060647 (+ 499us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.060694 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":99,"threads_started":1}
I20260809 02:15:00.061054 17186 tablet_service.cc:1467] Processing CreateTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 (DEFAULT_TABLE table=test-table [id=a86eed245105497fbb8ad4c128db446a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:00.061440 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.056079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41078 (local address 127.9.40.66:35527)
0809 02:15:00.056258 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.056263 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.056274 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:00.056323 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.056325 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.056341 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.056432 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.056561 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.057773 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.059107 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.059840 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.060352 (+ 512us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.060435 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.061037 (+ 602us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.061086 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":100,"threads_started":1}
I20260809 02:15:00.061460 17186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b0f522acb84f7c8f18b4cd23777186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:00.061769 17485 tablet_service.cc:1467] Processing CreateTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 (DEFAULT_TABLE table=test-table [id=a86eed245105497fbb8ad4c128db446a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:00.062144 17485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b0f522acb84f7c8f18b4cd23777186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:00.062054 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 (DEFAULT_TABLE table=test-table [id=a86eed245105497fbb8ad4c128db446a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:00.062381 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41b0f522acb84f7c8f18b4cd23777186. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:00.064529 17587 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Bootstrap starting.
I20260809 02:15:00.065220 17587 tablet_bootstrap.cc:654] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:00.065568 17587 log.cc:824] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:00.065735 17588 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Bootstrap starting.
I20260809 02:15:00.066004 17590 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Bootstrap starting.
I20260809 02:15:00.066372 17588 tablet_bootstrap.cc:654] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:00.066654 17590 tablet_bootstrap.cc:654] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:00.066664 17588 log.cc:824] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:00.066778 17587 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: No bootstrap required, opened a new log
I20260809 02:15:00.066857 17587 ts_tablet_manager.cc:1397] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:00.066991 17590 log.cc:824] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:00.067770 17590 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: No bootstrap required, opened a new log
I20260809 02:15:00.067840 17590 ts_tablet_manager.cc:1397] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:00.069082 17587 raft_consensus.cc:348] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.069293 17587 raft_consensus.cc:374] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:00.069656 17587 raft_consensus.cc:729] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d452ec7a4940c39cea6b49ab41a406, State: Initialized, Role: FOLLOWER
I20260809 02:15:00.069818 17587 consensus_queue.cc:260] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.070150 17587 ts_tablet_manager.cc:1428] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 02:15:00.070189 17263 heartbeater.cc:503] Master 127.9.40.126:37597 was elected leader, sending a full tablet report...
I20260809 02:15:00.071256 17588 tablet_bootstrap.cc:492] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: No bootstrap required, opened a new log
I20260809 02:15:00.071336 17588 ts_tablet_manager.cc:1397] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260809 02:15:00.071832 17583 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.068270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49770)
0809 02:15:00.068315 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.068324 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.068361 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.068567 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.068568 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.068570 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.068607 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.068609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.069405 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.069406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.070923 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.070926 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.071593 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.071594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.071651 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.071655 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.071712 (+ 57us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.071761 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260809 02:15:00.071899 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.068344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49770 (local address 127.9.40.124:46387)
0809 02:15:00.068412 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.068414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.068422 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:00.068449 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.068450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.068473 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.068548 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.068658 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.069214 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.071026 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.071498 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.071583 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.071652 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.071790 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.071830 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:15:00.072386 17074 raft_consensus.cc:1203] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.9] Dedup: 1.7->[1.8-1.9]
I20260809 02:15:00.072685 17590 raft_consensus.cc:348] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.072882 17590 raft_consensus.cc:374] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:00.072924 17590 raft_consensus.cc:729] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa991920d9a247419c0a95b7afeca1ca, State: Initialized, Role: FOLLOWER
I20260809 02:15:00.073055 17590 consensus_queue.cc:260] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.073221 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader dd39532293ad4e55ae39a0f51b8d1606. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:15:00.073307 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: LEARNER
I20260809 02:15:00.073460 17590 ts_tablet_manager.cc:1428] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Time spent starting tablet: real 0.006s user 0.004s sys 0.000s
I20260809 02:15:00.073596 17410 heartbeater.cc:503] Master 127.9.40.126:37597 was elected leader, sending a full tablet report...
I20260809 02:15:00.073565 17588 raft_consensus.cc:348] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.073716 17588 raft_consensus.cc:374] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:00.073760 17588 raft_consensus.cc:729] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddcde33c86445df9da586dc95035fae, State: Initialized, Role: FOLLOWER
I20260809 02:15:00.073874 17588 consensus_queue.cc:260] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.074143 17588 ts_tablet_manager.cc:1428] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:15:00.074451 17560 heartbeater.cc:503] Master 127.9.40.126:37597 was elected leader, sending a full tablet report...
W20260809 02:15:00.140769 17412 tablet.cc:2369] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:00.236155 17265 tablet.cc:2369] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:00.277967 17563 tablet.cc:2369] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:00.402896 17594 raft_consensus.cc:482] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:00.403079 17594 raft_consensus.cc:504] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.403466 17594 leader_election.cc:288] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527), 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:15:00.406648 17602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.403653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35527 (local address 127.9.40.65:39721)
0809 02:15:00.403794 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.403805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.403898 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.404077 (+ 179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.404078 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.404080 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.404164 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.404170 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.404926 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.404927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.405742 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.405744 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.406370 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.406371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.406450 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.406455 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.406515 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.406554 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"mutex_wait_us":20,"thread_start_us":74,"threads_started":1}
I20260809 02:15:00.406694 17584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.403740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39721 (local address 127.9.40.66:35527)
0809 02:15:00.403794 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.403797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.403874 (+ 77us) server_negotiation.cc:407] Connection header received
0809 02:15:00.403976 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.403977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.403988 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.404070 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.404253 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.404829 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.405888 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.406288 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.406346 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.406354 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.406531 (+ 177us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.406588 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 02:15:00.407078 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b0f522acb84f7c8f18b4cd23777186" candidate_uuid: "d5d452ec7a4940c39cea6b49ab41a406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa991920d9a247419c0a95b7afeca1ca" is_pre_election: true
I20260809 02:15:00.407143 17601 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.403684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35429 (local address 127.9.40.65:38789)
0809 02:15:00.403812 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:00.403822 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:00.403893 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:00.404074 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:00.404075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:00.404076 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:00.404159 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.404162 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.404901 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.404902 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.405741 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:00.405743 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.406906 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.406906 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:00.406967 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.406970 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:00.407021 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:15:00.407055 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"mutex_wait_us":31,"thread_start_us":63,"threads_started":1}
I20260809 02:15:00.407177 17586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.403743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:38789 (local address 127.9.40.67:35429)
0809 02:15:00.403794 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.403797 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.403906 (+ 109us) server_negotiation.cc:407] Connection header received
0809 02:15:00.403986 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.403987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.403998 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.404068 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:00.404241 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.404817 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.405841 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.406796 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.406861 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.406872 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.407047 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.407096 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:15:00.407244 17357 raft_consensus.cc:2454] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d452ec7a4940c39cea6b49ab41a406 in term 0.
I20260809 02:15:00.407517 17141 leader_election.cc:302] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d452ec7a4940c39cea6b49ab41a406, fa991920d9a247419c0a95b7afeca1ca; no voters:
I20260809 02:15:00.407531 17507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b0f522acb84f7c8f18b4cd23777186" candidate_uuid: "d5d452ec7a4940c39cea6b49ab41a406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddcde33c86445df9da586dc95035fae" is_pre_election: true
I20260809 02:15:00.407677 17507 raft_consensus.cc:2454] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5d452ec7a4940c39cea6b49ab41a406 in term 0.
I20260809 02:15:00.407686 17594 raft_consensus.cc:2781] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:00.407742 17594 raft_consensus.cc:482] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:00.407779 17594 raft_consensus.cc:3037] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:00.408548 17594 raft_consensus.cc:504] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.408715 17594 leader_election.cc:288] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [CANDIDATE]: Term 1 election: Requested vote from peers fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527), 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429)
I20260809 02:15:00.408865 17357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b0f522acb84f7c8f18b4cd23777186" candidate_uuid: "d5d452ec7a4940c39cea6b49ab41a406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fa991920d9a247419c0a95b7afeca1ca"
I20260809 02:15:00.408895 17507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41b0f522acb84f7c8f18b4cd23777186" candidate_uuid: "d5d452ec7a4940c39cea6b49ab41a406" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddcde33c86445df9da586dc95035fae"
I20260809 02:15:00.408977 17357 raft_consensus.cc:3037] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:00.408978 17507 raft_consensus.cc:3037] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:00.409667 17357 raft_consensus.cc:2454] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d452ec7a4940c39cea6b49ab41a406 in term 1.
I20260809 02:15:00.409842 17507 raft_consensus.cc:2454] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5d452ec7a4940c39cea6b49ab41a406 in term 1.
I20260809 02:15:00.409878 17141 leader_election.cc:302] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d452ec7a4940c39cea6b49ab41a406, fa991920d9a247419c0a95b7afeca1ca; no voters:
I20260809 02:15:00.409981 17594 raft_consensus.cc:2781] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:00.410102 17594 raft_consensus.cc:686] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 LEADER]: Becoming Leader. State: Replica: d5d452ec7a4940c39cea6b49ab41a406, State: Running, Role: LEADER
I20260809 02:15:00.410209 17594 consensus_queue.cc:237] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } }
I20260809 02:15:00.411019 16188 catalog_manager.cc:5495] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 reported cstate change: term changed from 0 to 1, leader changed from <none> to d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "d5d452ec7a4940c39cea6b49ab41a406" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:00.411677 16812 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: NON_VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: true } } }
I20260809 02:15:00.411777 16812 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:00.412518 16812 raft_consensus.cc:1050] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: attempting to promote NON_VOTER a79632bd01b4471c8ec2403909d485ec to VOTER
I20260809 02:15:00.412647 16812 consensus_queue.cc:237] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 10, Committed index: 10, Last appended: 1.10, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } }
I20260809 02:15:00.413170 16865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 02:15:00.413269 17074 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 LEARNER]: Refusing update from remote peer dd39532293ad4e55ae39a0f51b8d1606: Log matching property violated. Preceding OpId in replica: term: 1 index: 10. Preceding OpId from leader: term: 1 index: 11. (index mismatch)
I20260809 02:15:00.413573 16812 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 02:15:00.413688 17605 consensus_queue.cc:1035] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 11, Last known committed idx: 10, Time since last communication: 0.000s
I20260809 02:15:00.414582 17022 raft_consensus.cc:2932] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [term 1 LEADER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a79632bd01b4471c8ec2403909d485ec (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.414872 17074 raft_consensus.cc:2932] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a79632bd01b4471c8ec2403909d485ec (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.415519 16812 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.415604 16812 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:00.415643 17022 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.415745 17022 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:00.415659 16812 sys_catalog.cc:455] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.415882 16812 sys_catalog.cc:458] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:00.415802 16865 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Committing config change with OpId 1.11: config changed from index 7 to 11, a79632bd01b4471c8ec2403909d485ec (127.9.40.124) changed from NON_VOTER to VOTER. New config: { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.415911 17106 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.416082 17106 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:15:00.416652 17608 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "dd39532293ad4e55ae39a0f51b8d1606" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:00.416807 17608 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:15:00.503436 17585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.500993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41106 (local address 127.9.40.65:40255)
0809 02:15:00.501056 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.501058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.501068 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:00.501121 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.501122 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.501143 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.501235 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:00.501353 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.501834 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.502623 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.502821 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.502901 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.503072 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:00.503177 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:00.503272 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.503296 (+ 24us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.503316 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:15:00.505415 17507 raft_consensus.cc:1261] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Refusing update from remote peer d5d452ec7a4940c39cea6b49ab41a406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:00.505440 17357 raft_consensus.cc:1261] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Refusing update from remote peer d5d452ec7a4940c39cea6b49ab41a406: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:00.505801 17594 consensus_queue.cc:1035] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:00.505957 17603 consensus_queue.cc:1035] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:00.509181 17610 mvcc.cc:205] Tried to move back new op lower bound from 7316446005266952192 to 7316446004880764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316446005266952192}]
I20260809 02:15:00.509835 17612 mvcc.cc:205] Tried to move back new op lower bound from 7316446005266952192 to 7316446004880764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316446005266952192}]
I20260809 02:15:00.510277 17611 mvcc.cc:205] Tried to move back new op lower bound from 7316446005266952192 to 7316446004880764928. Current Snapshot: MvccSnapshot[applied={T|T < 7316446005266952192}]
I20260809 02:15:00.511693 16188 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:00.511809 16188 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45694:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:00.513568 16188 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dd39532293ad4e55ae39a0f51b8d1606: Sending DeleteTablet for 3 replicas of tablet 41b0f522acb84f7c8f18b4cd23777186
I20260809 02:15:00.513909 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:41078
I20260809 02:15:00.514415 17186 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:41090
I20260809 02:15:00.514478 17485 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58136
I20260809 02:15:00.514663 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 41b0f522acb84f7c8f18b4cd23777186 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:00 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58136
I20260809 02:15:00.514691 17621 tablet_replica.cc:330] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: stopping tablet replica
I20260809 02:15:00.514791 17621 raft_consensus.cc:2229] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:00.514907 17621 raft_consensus.cc:2258] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:00.515339 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16155
I20260809 02:15:00.521284 17621 ts_tablet_manager.cc:1905] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:00.523463 17621 ts_tablet_manager.cc:1918] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:00.523542 17621 log.cc:1196] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/41b0f522acb84f7c8f18b4cd23777186
I20260809 02:15:00.523816 17621 ts_tablet_manager.cc:1939] T 41b0f522acb84f7c8f18b4cd23777186 P fa991920d9a247419c0a95b7afeca1ca: Deleting consensus metadata
I20260809 02:15:00.524849 17624 tablet_replica.cc:330] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: stopping tablet replica
W20260809 02:15:00.524926 17288 connection.cc:535] server connection from 127.0.0.1:41078 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:15:00.524986 17624 raft_consensus.cc:2229] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:00.525102 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.514785 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49786 (local address 127.9.40.124:46387)
0809 02:15:00.514844 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.514847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.514855 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:00.514901 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.514902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.514912 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.514971 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.517930 (+ 2959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.518761 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.519882 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.520194 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.520279 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.521999 (+ 1720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.522001 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.522001 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.522009 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.523770 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.524183 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.524184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.524185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.524244 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.524450 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.524451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.524451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.524583 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.524671 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.524928 (+ 257us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.525016 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:15:00.525108 17624 raft_consensus.cc:2258] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:15:00.525192 16838 connection.cc:535] server connection from 127.0.0.1:36322 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:15:00.525647 17624 ts_tablet_manager.cc:1905] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:00.527112 17623 tablet_replica.cc:330] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: stopping tablet replica
I20260809 02:15:00.527256 17623 raft_consensus.cc:2229] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:00.527277 17624 ts_tablet_manager.cc:1918] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:00.527385 17623 raft_consensus.cc:2258] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:00.527448 17624 log.cc:1196] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/41b0f522acb84f7c8f18b4cd23777186
I20260809 02:15:00.527818 17624 ts_tablet_manager.cc:1939] T 41b0f522acb84f7c8f18b4cd23777186 P 8ddcde33c86445df9da586dc95035fae: Deleting consensus metadata
I20260809 02:15:00.527866 17623 ts_tablet_manager.cc:1905] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:15:00.528360 17435 connection.cc:441] server connection from 127.0.0.1:58136 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58136 (request call id 1) could send its response
W20260809 02:15:00.528565 17435 connection.cc:441] server connection from 127.0.0.1:58136 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:58136 (request call id 2) could send its response
I20260809 02:15:00.529809 17623 ts_tablet_manager.cc:1918] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:00.529917 17623 log.cc:1196] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/41b0f522acb84f7c8f18b4cd23777186
I20260809 02:15:00.530227 17623 ts_tablet_manager.cc:1939] T 41b0f522acb84f7c8f18b4cd23777186 P d5d452ec7a4940c39cea6b49ab41a406: Deleting consensus metadata
W20260809 02:15:00.530732 17138 connection.cc:441] server connection from 127.0.0.1:41090 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:41090 (request call id 1) could send its response
I20260809 02:15:00.536278 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.527301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36348 (local address 127.9.40.125:33343)
0809 02:15:00.527360 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.527363 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.527719 (+ 356us) server_negotiation.cc:407] Connection header received
0809 02:15:00.527985 (+ 266us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.527987 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.528003 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.528034 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.528871 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.529642 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.530734 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.531054 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.531095 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.533059 (+ 1964us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.533061 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.533061 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.533068 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.534891 (+ 1823us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.535337 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.535339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.535339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.535403 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.535626 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.535627 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.535627 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.535768 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.535844 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.536053 (+ 209us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.536163 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:15:00.536639 17268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:00.527494 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49800 (local address 127.9.40.124:46387)
0809 02:15:00.527596 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:00.527598 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:00.528074 (+ 476us) server_negotiation.cc:407] Connection header received
0809 02:15:00.528190 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:00.528192 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:00.528202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:00.528234 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:00.529810 (+ 1576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.530621 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.531686 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:00.532002 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:00.532088 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:00.533461 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:00.533463 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:00.533463 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:00.533471 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:00.535331 (+ 1860us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.535669 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.535671 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.535671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.535731 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:00.535967 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:00.535968 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:00.535969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:00.536106 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:00.536215 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:00.536444 (+ 229us) server_negotiation.cc:299] Negotiation successful
0809 02:15:00.536543 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260809 02:15:01.512152 17631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:01.511856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:36579)
0809 02:15:01.511998 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:01.512077 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":81,"threads_started":1}
I20260809 02:15:01.512226 17632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:01.511891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:48455)
0809 02:15:01.512079 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:01.512164 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":145,"threads_started":1}
W20260809 02:15:01.512403 17410 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:37597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
W20260809 02:15:01.512502 17263 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:37597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:01.515100 17633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:01.514879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:46379)
0809 02:15:01.514996 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:01.515061 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":63,"threads_started":1}
W20260809 02:15:01.515375 17560 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:37597 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.014181 17634 raft_consensus.cc:482] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd39532293ad4e55ae39a0f51b8d1606)
I20260809 02:15:02.014354 17634 raft_consensus.cc:504] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } }
I20260809 02:15:02.014673 17634 leader_election.cc:288] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597), 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125:33343)
I20260809 02:15:02.015148 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.014821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46182)
0809 02:15:02.015027 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.015091 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":68,"threads_started":1}
W20260809 02:15:02.015344 17045 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.015419 17638 raft_consensus.cc:482] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader dd39532293ad4e55ae39a0f51b8d1606)
I20260809 02:15:02.015534 17638 raft_consensus.cc:504] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } }
I20260809 02:15:02.015799 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.015698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46194)
0809 02:15:02.015737 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.015773 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":20}
I20260809 02:15:02.015799 17638 leader_election.cc:288] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597), a79632bd01b4471c8ec2403909d485ec (127.9.40.124:46387)
W20260809 02:15:02.016001 17045 leader_election.cc:334] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597): Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.016222 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.015865 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46202)
0809 02:15:02.016115 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.016176 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":96,"threads_started":1}
W20260809 02:15:02.016315 16839 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.016794 17640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.016657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46216)
0809 02:15:02.016714 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.016769 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 02:15:02.016942 16839 leader_election.cc:334] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597): Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.018630 17637 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.015128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36350 (local address 127.9.40.125:33343)
0809 02:15:02.015324 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.015327 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.015336 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:02.015367 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.015368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.015379 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.015441 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.015678 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.016468 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.017734 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.018281 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.018356 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.018371 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.018525 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.018567 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":78,"threads_started":1}
I20260809 02:15:02.018641 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.015009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:33343 (local address 127.0.0.1:36350)
0809 02:15:02.015130 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.015144 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.015223 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.015505 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.015505 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.015508 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.015594 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.015597 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.016573 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.016574 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.017641 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.017644 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.018357 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.018358 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.018426 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.018430 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.018515 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.018563 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260809 02:15:02.018985 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a79632bd01b4471c8ec2403909d485ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "45fbacfb7e304be99f48e7d0c9660e63" is_pre_election: true
I20260809 02:15:02.019121 16865 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a79632bd01b4471c8ec2403909d485ec in term 1.
I20260809 02:15:02.019273 17045 leader_election.cc:302] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 45fbacfb7e304be99f48e7d0c9660e63, a79632bd01b4471c8ec2403909d485ec; no voters: dd39532293ad4e55ae39a0f51b8d1606
I20260809 02:15:02.019423 17634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:15:02.019461 17634 raft_consensus.cc:482] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Starting leader election (detected failure of leader dd39532293ad4e55ae39a0f51b8d1606)
I20260809 02:15:02.019498 17634 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:15:02.019547 17641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.016072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46387 (local address 127.0.0.1:49810)
0809 02:15:02.016165 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.016175 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.016223 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.016521 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.016522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.016524 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.016597 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.016599 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.017574 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.017575 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.018624 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.018626 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.019251 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.019252 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.019316 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.019319 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.019410 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.019454 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":25,"thread_start_us":51,"threads_started":1}
I20260809 02:15:02.019574 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.016231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49810 (local address 127.9.40.124:46387)
0809 02:15:02.016359 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.016361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.016368 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:15:02.016392 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.016393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.016405 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.016667 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.016706 (+ 39us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.017487 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.018710 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.019181 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.019249 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.019259 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.019421 (+ 162us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.019464 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":101,"thread_start_us":78,"threads_started":1}
I20260809 02:15:02.019805 17074 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "45fbacfb7e304be99f48e7d0c9660e63" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "a79632bd01b4471c8ec2403909d485ec" is_pre_election: true
I20260809 02:15:02.020370 17634 raft_consensus.cc:504] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 2 FOLLOWER]: Starting leader election with config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } }
I20260809 02:15:02.020524 17634 leader_election.cc:288] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 election: Requested vote from peers dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597), 45fbacfb7e304be99f48e7d0c9660e63 (127.9.40.125:33343)
I20260809 02:15:02.020566 17074 raft_consensus.cc:2379] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 45fbacfb7e304be99f48e7d0c9660e63 in current term 2: Already voted for candidate a79632bd01b4471c8ec2403909d485ec in this term.
I20260809 02:15:02.020773 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.020624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46230)
0809 02:15:02.020693 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.020745 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260809 02:15:02.020788 16865 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "a79632bd01b4471c8ec2403909d485ec" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "45fbacfb7e304be99f48e7d0c9660e63"
I20260809 02:15:02.020820 16839 leader_election.cc:302] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 45fbacfb7e304be99f48e7d0c9660e63; no voters: a79632bd01b4471c8ec2403909d485ec, dd39532293ad4e55ae39a0f51b8d1606
I20260809 02:15:02.020854 16865 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:15:02.021459 16865 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a79632bd01b4471c8ec2403909d485ec in term 2.
I20260809 02:15:02.021541 17638 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260809 02:15:02.021785 17045 leader_election.cc:302] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 45fbacfb7e304be99f48e7d0c9660e63, a79632bd01b4471c8ec2403909d485ec; no voters:
I20260809 02:15:02.021826 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.021744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46234)
0809 02:15:02.021778 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.021805 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 02:15:02.021863 17045 leader_election.cc:334] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597): Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
I20260809 02:15:02.021906 17634 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:15:02.022075 17634 raft_consensus.cc:686] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [term 2 LEADER]: Becoming Leader. State: Replica: a79632bd01b4471c8ec2403909d485ec, State: Running, Role: LEADER
I20260809 02:15:02.022178 17634 consensus_queue.cc:237] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } }
I20260809 02:15:02.022591 17645 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader a79632bd01b4471c8ec2403909d485ec. Latest consensus state: current_term: 2 leader_uuid: "a79632bd01b4471c8ec2403909d485ec" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:02.022675 17645 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:02.022838 17647 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:02.023460 17647 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:02.023538 17647 catalog_manager.cc:1259] Loaded cluster ID: df6bed2e7236435094129a9bfbaea2ee
I20260809 02:15:02.023562 17647 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:02.023707 17647 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:02.023761 17647 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Loaded TSK: 0
I20260809 02:15:02.023862 17647 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:15:02.026695 17063 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49800:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:02.028389 17063 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:02.029464 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.029352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46252)
0809 02:15:02.029406 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.029437 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:15:02.029599 16865 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [term 2 FOLLOWER]: Refusing update from remote peer a79632bd01b4471c8ec2403909d485ec: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260809 02:15:02.029922 17634 consensus_queue.cc:1035] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [LEADER]: Connected to new peer: Peer: permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260809 02:15:02.030110 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.029973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.0.0.1:46264)
0809 02:15:02.030023 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.030082 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 02:15:02.030187 17045 consensus_peers.cc:487] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec -> Peer dd39532293ad4e55ae39a0f51b8d1606 (127.9.40.126:37597): Couldn't send request to peer dd39532293ad4e55ae39a0f51b8d1606. Status: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260809 02:15:02.030658 17638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a79632bd01b4471c8ec2403909d485ec. Latest consensus state: current_term: 2 leader_uuid: "a79632bd01b4471c8ec2403909d485ec" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:02.030758 17638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:15:02.031183 17645 sys_catalog.cc:455] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "a79632bd01b4471c8ec2403909d485ec" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:02.031277 17645 sys_catalog.cc:458] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:02.031378 17638 sys_catalog.cc:455] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "a79632bd01b4471c8ec2403909d485ec" committed_config { opid_index: 11 OBSOLETE_local: true peers { permanent_uuid: "dd39532293ad4e55ae39a0f51b8d1606" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37597 } } peers { permanent_uuid: "45fbacfb7e304be99f48e7d0c9660e63" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 33343 } attrs { promote: false } } peers { permanent_uuid: "a79632bd01b4471c8ec2403909d485ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46387 } attrs { promote: false } } }
I20260809 02:15:02.031467 17638 sys_catalog.cc:458] T 00000000000000000000000000000000 P 45fbacfb7e304be99f48e7d0c9660e63 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:15:02.036506 17636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35429 (local address 127.0.0.1:58152)
0809 02:15:02.033312 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.033324 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.033377 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.033637 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.033638 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.033639 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.033693 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.033695 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.034406 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.034407 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.035478 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.035480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.036253 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.036254 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.036318 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.036324 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.036389 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.036430 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260809 02:15:02.036528 17654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58152 (local address 127.9.40.67:35429)
0809 02:15:02.033476 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.033478 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.033488 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:02.033528 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.033529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.033540 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.033760 (+ 220us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.033803 (+ 43us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.034327 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.035588 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.036164 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.036238 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.036251 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.036407 (+ 156us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.036450 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":75,"threads_started":1}
I20260809 02:15:02.036837 17484 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 (DEFAULT_TABLE table=test-table [id=55a328a5df5a47d384b358967c95dab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:02.037003 17484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c9788eb15408d8c80a776f01ae2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:02.037626 17655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033363 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40255 (local address 127.0.0.1:41112)
0809 02:15:02.034430 (+ 1067us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.034440 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.034474 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.034860 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.034860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.034862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.034895 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.034896 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.035499 (+ 603us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.035500 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.036686 (+ 1186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.036689 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.037329 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.037330 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.037403 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.037408 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.037490 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.037537 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1044,"mutex_wait_us":40,"thread_start_us":90,"threads_started":1}
I20260809 02:15:02.037618 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41112 (local address 127.9.40.65:40255)
0809 02:15:02.034755 (+ 1178us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.034757 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.034765 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:02.034791 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.034792 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.034802 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.034860 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.034946 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.035411 (+ 465us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.036813 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.037222 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.037325 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.037338 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.037498 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.037547 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1153,"thread_start_us":61,"threads_started":1}
I20260809 02:15:02.037853 17186 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 (DEFAULT_TABLE table=test-table [id=55a328a5df5a47d384b358967c95dab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:02.038026 17186 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c9788eb15408d8c80a776f01ae2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:02.038373 17635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033305 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35527 (local address 127.0.0.1:41084)
0809 02:15:02.033416 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.033436 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.033489 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.034670 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.034671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.034674 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.034715 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.034716 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.035685 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.035686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.036666 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.036669 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.038136 (+ 1467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.038137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.038206 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.038209 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.038273 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.038314 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":94}
I20260809 02:15:02.038412 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.033575 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41084 (local address 127.9.40.66:35527)
0809 02:15:02.034551 (+ 976us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.034552 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.034558 (+ 6us) server_negotiation.cc:407] Connection header received
0809 02:15:02.034582 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.034582 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.034591 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.034668 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.034812 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.035528 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.037145 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.038019 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.038098 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.038113 (+ 15us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.038294 (+ 181us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.038337 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":956,"thread_start_us":68,"threads_started":1}
I20260809 02:15:02.038616 17335 tablet_service.cc:1467] Processing CreateTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 (DEFAULT_TABLE table=test-table [id=55a328a5df5a47d384b358967c95dab0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:02.038730 17335 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0e8c9788eb15408d8c80a776f01ae2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:02.039048 17658 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Bootstrap starting.
I20260809 02:15:02.039549 17658 tablet_bootstrap.cc:654] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:02.039855 17659 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: Bootstrap starting.
I20260809 02:15:02.040216 17659 tablet_bootstrap.cc:654] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:02.040243 17658 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: No bootstrap required, opened a new log
I20260809 02:15:02.040285 17658 ts_tablet_manager.cc:1397] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:02.040426 17658 raft_consensus.cc:348] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.040513 17658 raft_consensus.cc:374] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:02.040529 17658 raft_consensus.cc:729] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8ddcde33c86445df9da586dc95035fae, State: Initialized, Role: FOLLOWER
I20260809 02:15:02.040628 17658 consensus_queue.cc:260] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.040802 17662 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Bootstrap starting.
I20260809 02:15:02.040835 17658 ts_tablet_manager.cc:1428] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:02.040827 17659 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: No bootstrap required, opened a new log
I20260809 02:15:02.040892 17659 ts_tablet_manager.cc:1397] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:02.041025 17659 raft_consensus.cc:348] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.041097 17659 raft_consensus.cc:374] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:02.041119 17659 raft_consensus.cc:729] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5d452ec7a4940c39cea6b49ab41a406, State: Initialized, Role: FOLLOWER
I20260809 02:15:02.041185 17662 tablet_bootstrap.cc:654] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:02.041172 17659 consensus_queue.cc:260] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.041328 17659 ts_tablet_manager.cc:1428] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:15:02.041661 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.041266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:44363)
0809 02:15:02.041572 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.041615 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":86,"threads_started":1}
I20260809 02:15:02.041723 17559 heartbeater.cc:503] Master 127.9.40.124:46387 was elected leader, sending a full tablet report...
I20260809 02:15:02.042101 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.041875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:40813)
0809 02:15:02.042049 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.042079 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 02:15:02.042413 17662 tablet_bootstrap.cc:492] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: No bootstrap required, opened a new log
I20260809 02:15:02.042476 17662 ts_tablet_manager.cc:1397] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:02.042666 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.042424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:47429)
0809 02:15:02.042589 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.042631 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":114,"threads_started":1}
I20260809 02:15:02.042651 17662 raft_consensus.cc:348] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.042727 17662 raft_consensus.cc:374] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:02.042752 17662 raft_consensus.cc:729] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa991920d9a247419c0a95b7afeca1ca, State: Initialized, Role: FOLLOWER
I20260809 02:15:02.042819 17662 consensus_queue.cc:260] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.042986 17662 ts_tablet_manager.cc:1428] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260809 02:15:02.043253 17262 heartbeater.cc:503] Master 127.9.40.124:46387 was elected leader, sending a full tablet report...
I20260809 02:15:02.043435 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.043179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:47655)
0809 02:15:02.043357 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.043395 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260809 02:15:02.043615 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.043371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:48287)
0809 02:15:02.043559 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.043587 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":169}
I20260809 02:15:02.043673 17409 heartbeater.cc:503] Master 127.9.40.124:46387 was elected leader, sending a full tablet report...
I20260809 02:15:02.142479 17671 raft_consensus.cc:482] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:02.142606 17671 raft_consensus.cc:504] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.142910 17671 leader_election.cc:288] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429), d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
I20260809 02:15:02.147199 17654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.143262 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:45477 (local address 127.9.40.67:35429)
0809 02:15:02.143324 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.143326 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.143338 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:15:02.143376 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.143377 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.143392 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.143496 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.143726 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.144471 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146178 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.146776 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146846 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.146859 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.147065 (+ 206us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.147131 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:15:02.147233 17656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.143431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42427 (local address 127.9.40.65:40255)
0809 02:15:02.143503 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.143505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.143618 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:15:02.143705 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.143706 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.143720 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.143785 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:02.144000 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.144666 (+ 666us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146180 (+ 1514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.146758 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146846 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.146859 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.147121 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.147166 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:15:02.147207 17672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.143156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35429 (local address 127.9.40.66:45477)
0809 02:15:02.143226 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.143243 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.143307 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.143548 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.143550 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.143553 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.143644 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.143646 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.144558 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.144559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.146039 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.146043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.146898 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.146975 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.146978 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.147042 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.147100 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:15:02.147215 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.143330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40255 (local address 127.9.40.66:42427)
0809 02:15:02.143542 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.143553 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.143619 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.143840 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:02.143841 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:02.143844 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:02.143908 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.143909 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.144769 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.144770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.146039 (+ 1269us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:02.146043 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.146865 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.146866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:02.146951 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.146956 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:15:02.147100 (+ 144us) client_negotiation.cc:240] Negotiation successful
0809 02:15:02.147132 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":149,"threads_started":1}
I20260809 02:15:02.147804 17507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8c9788eb15408d8c80a776f01ae2d2" candidate_uuid: "fa991920d9a247419c0a95b7afeca1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddcde33c86445df9da586dc95035fae" is_pre_election: true
I20260809 02:15:02.147828 17206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8c9788eb15408d8c80a776f01ae2d2" candidate_uuid: "fa991920d9a247419c0a95b7afeca1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d452ec7a4940c39cea6b49ab41a406" is_pre_election: true
I20260809 02:15:02.147918 17507 raft_consensus.cc:2454] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa991920d9a247419c0a95b7afeca1ca in term 0.
I20260809 02:15:02.147974 17206 raft_consensus.cc:2454] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa991920d9a247419c0a95b7afeca1ca in term 0.
I20260809 02:15:02.148200 17287 leader_election.cc:302] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8ddcde33c86445df9da586dc95035fae, fa991920d9a247419c0a95b7afeca1ca; no voters:
I20260809 02:15:02.148444 17671 raft_consensus.cc:2781] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:02.148514 17671 raft_consensus.cc:482] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:02.148542 17671 raft_consensus.cc:3037] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:02.149526 17671 raft_consensus.cc:504] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.149713 17671 leader_election.cc:288] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [CANDIDATE]: Term 1 election: Requested vote from peers 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429), d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255)
I20260809 02:15:02.149926 17507 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8c9788eb15408d8c80a776f01ae2d2" candidate_uuid: "fa991920d9a247419c0a95b7afeca1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8ddcde33c86445df9da586dc95035fae"
I20260809 02:15:02.150007 17206 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0e8c9788eb15408d8c80a776f01ae2d2" candidate_uuid: "fa991920d9a247419c0a95b7afeca1ca" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d5d452ec7a4940c39cea6b49ab41a406"
I20260809 02:15:02.150069 17507 raft_consensus.cc:3037] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:02.150077 17206 raft_consensus.cc:3037] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:02.150918 17507 raft_consensus.cc:2454] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa991920d9a247419c0a95b7afeca1ca in term 1.
I20260809 02:15:02.150976 17206 raft_consensus.cc:2454] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa991920d9a247419c0a95b7afeca1ca in term 1.
I20260809 02:15:02.151135 17287 leader_election.cc:302] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d5d452ec7a4940c39cea6b49ab41a406, fa991920d9a247419c0a95b7afeca1ca; no voters:
I20260809 02:15:02.151262 17671 raft_consensus.cc:2781] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:02.151448 17671 raft_consensus.cc:686] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 1 LEADER]: Becoming Leader. State: Replica: fa991920d9a247419c0a95b7afeca1ca, State: Running, Role: LEADER
I20260809 02:15:02.151543 17671 consensus_queue.cc:237] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } }
I20260809 02:15:02.151867 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.151754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:40543)
0809 02:15:02.151805 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.151836 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 02:15:02.152626 17062 catalog_manager.cc:5495] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca reported cstate change: term changed from 0 to 1, leader changed from <none> to fa991920d9a247419c0a95b7afeca1ca (127.9.40.66). New cstate: current_term: 1 leader_uuid: "fa991920d9a247419c0a95b7afeca1ca" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa991920d9a247419c0a95b7afeca1ca" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35527 } health_report { overall_health: HEALTHY } } }
I20260809 02:15:02.184530 17657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.181882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41086 (local address 127.9.40.66:35527)
0809 02:15:02.181954 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:02.181956 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:02.181965 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:02.182021 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:02.182021 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:02.182047 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:02.182138 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:02.182277 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.182770 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.183661 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:02.183854 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:02.183912 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:02.184142 (+ 230us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:02.184252 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:02.184333 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:02.184370 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 02:15:02.184398 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:15:02.185910 17507 raft_consensus.cc:1261] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Refusing update from remote peer fa991920d9a247419c0a95b7afeca1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:02.186086 17206 raft_consensus.cc:1261] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Refusing update from remote peer fa991920d9a247419c0a95b7afeca1ca: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:02.186362 17671 consensus_queue.cc:1035] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "8ddcde33c86445df9da586dc95035fae" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35429 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:02.186545 17674 consensus_queue.cc:1035] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [LEADER]: Connected to new peer: Peer: permanent_uuid: "d5d452ec7a4940c39cea6b49ab41a406" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 40255 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:02.187456 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.187312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:39187)
0809 02:15:02.187377 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.187419 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:15:02.187927 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.187747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:50815)
0809 02:15:02.187857 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.187894 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":82}
I20260809 02:15:02.189424 17673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.189256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.66:59629)
0809 02:15:02.189360 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.189392 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260809 02:15:02.189689 17668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.189240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.67:56321)
0809 02:15:02.189621 (+ 381us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.189654 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361}
I20260809 02:15:02.189761 17670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:02.189653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37597 (local address 127.9.40.65:55211)
0809 02:15:02.189688 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.189714 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:37597: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260809 02:15:02.192145 17062 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:02.192250 17062 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49800:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:02.193615 17062 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a79632bd01b4471c8ec2403909d485ec: Sending DeleteTablet for 3 replicas of tablet 0e8c9788eb15408d8c80a776f01ae2d2
I20260809 02:15:02.193893 17484 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:02 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:58152
I20260809 02:15:02.193948 17335 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:02 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:41084
I20260809 02:15:02.194116 17186 tablet_service.cc:1514] Processing DeleteTablet for tablet 0e8c9788eb15408d8c80a776f01ae2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:02 UTC) from {username='oryx', principal='oryx/127.9.40.124@KRBTEST.COM'} at 127.0.0.1:41112
I20260809 02:15:02.194213 17689 tablet_replica.cc:330] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: stopping tablet replica
I20260809 02:15:02.194285 17689 raft_consensus.cc:2229] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:02.194347 17689 raft_consensus.cc:2258] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:02.194437 17691 tablet_replica.cc:330] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406: stopping tablet replica
I20260809 02:15:02.194540 17690 tablet_replica.cc:330] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: stopping tablet replica
I20260809 02:15:02.194550 17691 raft_consensus.cc:2229] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:02.194609 17690 raft_consensus.cc:2229] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:02.194670 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17018
I20260809 02:15:02.194710 17690 raft_consensus.cc:2258] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:02.194723 17689 ts_tablet_manager.cc:1905] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:02.194820 17691 raft_consensus.cc:2258] T 0e8c9788eb15408d8c80a776f01ae2d2 P d5d452ec7a4940c39cea6b49ab41a406 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:02.197295 17689 ts_tablet_manager.cc:1918] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:02.197400 17689 log.cc:1196] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/0e8c9788eb15408d8c80a776f01ae2d2
I20260809 02:15:02.197666 17689 ts_tablet_manager.cc:1939] T 0e8c9788eb15408d8c80a776f01ae2d2 P 8ddcde33c86445df9da586dc95035fae: Deleting consensus metadata
I20260809 02:15:02.198133 17045 catalog_manager.cc:4841] TS 8ddcde33c86445df9da586dc95035fae (127.9.40.67:35429): tablet 0e8c9788eb15408d8c80a776f01ae2d2 (table test-table [id=55a328a5df5a47d384b358967c95dab0]) successfully deleted
I20260809 02:15:02.198747 17690 ts_tablet_manager.cc:1905] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:02.201135 17690 ts_tablet_manager.cc:1918] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:02.201208 17690 log.cc:1196] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/0e8c9788eb15408d8c80a776f01ae2d2
I20260809 02:15:02.201423 17690 ts_tablet_manager.cc:1939] T 0e8c9788eb15408d8c80a776f01ae2d2 P fa991920d9a247419c0a95b7afeca1ca: Deleting consensus metadata
I20260809 02:15:02.202044 17047 catalog_manager.cc:4841] TS fa991920d9a247419c0a95b7afeca1ca (127.9.40.66:35527): tablet 0e8c9788eb15408d8c80a776f01ae2d2 (table test-table [id=55a328a5df5a47d384b358967c95dab0]) successfully deleted
W20260809 02:15:02.212602 17635 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:02.212389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:40255 (local address 127.0.0.1:41126)
0809 02:15:02.212453 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:02.212464 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:02.212506 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:02.212552 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:40255: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:15:02.212807 17045 catalog_manager.cc:4584] TS d5d452ec7a4940c39cea6b49ab41a406 (127.9.40.65:40255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0e8c9788eb15408d8c80a776f01ae2d2: Network error: Client connection negotiation failed: client connection to 127.9.40.65:40255: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260809 02:15:02.212805 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17270
I20260809 02:15:02.219995 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17416
I20260809 02:15:02.227622 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16710
I20260809 02:15:02.234902 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 16912
2026-08-09T02:15:02Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6079 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:02 dist-test-slave-qg60 krb5kdc[17694](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:02 dist-test-slave-qg60 krb5kdc[17694](info): set up 2 sockets
Aug 09 02:15:02 dist-test-slave-qg60 krb5kdc[17694](info): commencing operation
krb5kdc: starting...
W20260809 02:15:03.369745 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.110s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:03Z Disabled control of system clock
WARNING: no policy specified for oryx/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
Entry for principal oryx/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab.
I20260809 02:15:03.501338 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33203
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33203
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:03.584815 17709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:03.585081 17709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:03.585139 17709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:03.586347 17709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:03.586421 17709 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260809 02:15:03.586442 17709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:03.586464 17709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:03.586493 17709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:03.586516 17709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:03.588052 17709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33203
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.126.keytab
--principal=oryx/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:03.588335 17709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:03.588568 17709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:03.593755 17717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:03.593765 17716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:03.593766 17719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:03.594002 17709 server_base.cc:1034] running on GCE node
I20260809 02:15:03.594357 17709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:03.594671 17709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:03.595836 17709 hybrid_clock.cc:648] HybridClock initialized: now 1786241703595817 us; error 44 us; skew 500 ppm
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:03.604444 17709 init.cc:377] Logged in from keytab as oryx/127.9.40.126@KRBTEST.COM (short username oryx)
I20260809 02:15:03.605034 17709 webserver.cc:460] Webserver started at http://127.9.40.126:37001/ using document root <none> and password file <none>
I20260809 02:15:03.605232 17709 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:03.605273 17709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:03.605376 17709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:03.606467 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "ac0f7671b22047cca1af257cad142f7e"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "e94dc1045a97aad9c7f4a62290c5bd90"
server_key_iv: "b4c2d3f02e49be6d5e9d613be90f6a75"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.606815 17709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "ac0f7671b22047cca1af257cad142f7e"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "e94dc1045a97aad9c7f4a62290c5bd90"
server_key_iv: "b4c2d3f02e49be6d5e9d613be90f6a75"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.608354 17709 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:03.609416 17727 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:03.609721 17709 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:03.609833 17709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "ac0f7671b22047cca1af257cad142f7e"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "e94dc1045a97aad9c7f4a62290c5bd90"
server_key_iv: "b4c2d3f02e49be6d5e9d613be90f6a75"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.609917 17709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:03.620371 17709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:03.622855 17709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:03.623028 17709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:03.628088 17709 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33203
I20260809 02:15:03.628134 17794 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33203 every 8 connection(s)
I20260809 02:15:03.628490 17709 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:03.629222 17795 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:03.631527 17795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Bootstrap starting.
I20260809 02:15:03.632112 17795 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:03.632362 17795 log.cc:824] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:03.633152 17795 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: No bootstrap required, opened a new log
I20260809 02:15:03.635243 17795 raft_consensus.cc:348] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } }
I20260809 02:15:03.635390 17795 raft_consensus.cc:374] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:03.635412 17795 raft_consensus.cc:729] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac0f7671b22047cca1af257cad142f7e, State: Initialized, Role: FOLLOWER
I20260809 02:15:03.635608 17795 consensus_queue.cc:260] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } }
I20260809 02:15:03.635710 17795 raft_consensus.cc:388] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:03.635735 17795 raft_consensus.cc:482] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:03.635764 17795 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:03.636338 17795 raft_consensus.cc:504] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } }
I20260809 02:15:03.636427 17795 leader_election.cc:302] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ac0f7671b22047cca1af257cad142f7e; no voters:
I20260809 02:15:03.636565 17795 leader_election.cc:288] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:03.636641 17799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:03.636848 17799 raft_consensus.cc:686] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [term 1 LEADER]: Becoming Leader. State: Replica: ac0f7671b22047cca1af257cad142f7e, State: Running, Role: LEADER
I20260809 02:15:03.636899 17795 sys_catalog.cc:564] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:03.637025 17799 consensus_queue.cc:237] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } }
I20260809 02:15:03.637437 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17709
I20260809 02:15:03.637565 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:15:03.637813 9377 external_mini_cluster.cc:1428] Setting key c367eb2e70bd80f3edde8c08baef97ba
I20260809 02:15:03.638046 17800 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ac0f7671b22047cca1af257cad142f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } } }
I20260809 02:15:03.638134 17800 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:03.638293 17801 sys_catalog.cc:455] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [sys.catalog]: SysCatalogTable state changed. Reason: New leader ac0f7671b22047cca1af257cad142f7e. Latest consensus state: current_term: 1 leader_uuid: "ac0f7671b22047cca1af257cad142f7e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ac0f7671b22047cca1af257cad142f7e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33203 } } }
I20260809 02:15:03.638367 17801 sys_catalog.cc:458] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e [sys.catalog]: This master's current role is: LEADER
W20260809 02:15:03.638965 17818 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:15:03.639026 17818 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:15:03.639072 17814 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:03.639238 17814 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:03.640817 17814 catalog_manager.cc:1347] Generated new cluster ID: 6cd28069831a422db8973a19d26e025d
I20260809 02:15:03.640874 17814 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:03.648305 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:03.638513 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59266 (local address 127.9.40.126:33203)
0809 02:15:03.638666 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:03.638677 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:03.638714 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:15:03.638845 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:03.638852 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:03.638908 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:03.639185 (+ 277us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:03.639809 (+ 624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.641147 (+ 1338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.641915 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.642348 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.642481 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:03.644913 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:03.644917 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:03.644918 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:03.644930 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:03.646663 (+ 1733us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.647075 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.647078 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.647078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.647164 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.647381 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.647382 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.647382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.647656 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:03.647830 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:03.647941 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:15:03.648091 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":101,"thread_start_us":75,"threads_started":1}
I20260809 02:15:03.656373 17814 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:03.657169 17814 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:03.666085 17814 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Generated new TSK 0
I20260809 02:15:03.666303 17814 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
I20260809 02:15:03.704805 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33203
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:03.797199 17824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:03.797434 17824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:03.797461 17824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:03.798610 17824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:03.798650 17824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:03.798669 17824 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:03.798686 17824 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:03.800194 17824 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:03.800413 17824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:03.800648 17824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:03.806130 17829 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:03.806159 17832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:03.806190 17830 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:03.806334 17824 server_base.cc:1034] running on GCE node
I20260809 02:15:03.806622 17824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:03.806903 17824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:03.808087 17824 hybrid_clock.cc:648] HybridClock initialized: now 1786241703808067 us; error 52 us; skew 500 ppm
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:03.810973 17824 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260809 02:15:03.811560 17824 webserver.cc:460] Webserver started at http://127.9.40.65:42163/ using document root <none> and password file <none>
I20260809 02:15:03.811798 17824 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:03.811872 17824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:03.812019 17824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:03.813289 17824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "bd5136614a8d4f9082c100c0525324e9"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "337dc09e89722392072054622190ef72"
server_key_iv: "7bb8408ce53ac030ba95871a1d212b3b"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.813673 17824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "bd5136614a8d4f9082c100c0525324e9"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "337dc09e89722392072054622190ef72"
server_key_iv: "7bb8408ce53ac030ba95871a1d212b3b"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.815376 17824 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:15:03.816264 17839 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:03.816458 17824 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:03.816560 17824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "bd5136614a8d4f9082c100c0525324e9"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "337dc09e89722392072054622190ef72"
server_key_iv: "7bb8408ce53ac030ba95871a1d212b3b"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.816653 17824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:03.839360 17824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:03.841840 17824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:03.842012 17824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:03.842634 17824 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:03.842695 17824 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:03.842737 17824 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:03.842760 17824 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:03.850723 17824 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:39083
I20260809 02:15:03.850790 17965 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:39083 every 8 connection(s)
I20260809 02:15:03.851176 17824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:03.851455 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17824
I20260809 02:15:03.851567 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:03.851804 9377 external_mini_cluster.cc:1428] Setting key 1957eab4a35809b82d0a7e480bbac558
I20260809 02:15:03.853755 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:03.843166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51001 (local address 127.9.40.126:33203)
0809 02:15:03.843243 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:03.843246 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:03.843425 (+ 179us) server_negotiation.cc:407] Connection header received
0809 02:15:03.843937 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:03.843939 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:03.843957 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:03.844047 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:03.844815 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.845337 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.846437 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.846710 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.846785 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:03.849939 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:03.849941 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:03.849941 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:03.849951 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:03.851895 (+ 1944us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.852406 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.852407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.852407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.852520 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.853007 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.853008 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.853008 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.853165 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:03.853241 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:03.853511 (+ 270us) server_negotiation.cc:299] Negotiation successful
0809 02:15:03.853611 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:15:03.853770 17848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:03.843133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.65:51001)
0809 02:15:03.843289 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:03.843329 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:03.843833 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:03.844136 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:03.844141 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:03.844437 (+ 296us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:03.844683 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:03.844689 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.845440 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.845441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:03.846318 (+ 877us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:03.846321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.846806 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.846807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:03.846937 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:03.847523 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:03.847545 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:03.849815 (+ 2270us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:03.851994 (+ 2179us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:03.852000 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:03.852002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:03.852337 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:03.852761 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:03.852761 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:03.852762 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:03.852915 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:03.853239 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:03.853244 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:03.853366 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 02:15:03.853445 (+ 79us) client_negotiation.cc:240] Negotiation successful
0809 02:15:03.853611 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 02:15:03.854485 17966 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33203
I20260809 02:15:03.854581 17966 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:03.854803 17966 heartbeater.cc:511] Master 127.9.40.126:33203 requested a full tablet report, sending...
I20260809 02:15:03.855304 17743 ts_manager.cc:194] Registered new tserver with Master: bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083)
I20260809 02:15:03.855923 17743 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:51001
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
I20260809 02:15:03.866925 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:03.860355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40765 (local address 127.9.40.126:33203)
0809 02:15:03.860402 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:03.860408 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:03.860523 (+ 115us) server_negotiation.cc:407] Connection header received
0809 02:15:03.860602 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:03.860603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:03.860611 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:03.860659 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:03.861122 (+ 463us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.861650 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.862384 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.862581 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.862642 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:03.863670 (+ 1028us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:03.863671 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:03.863672 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:03.863703 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:03.865704 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.866051 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.866052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.866053 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.866116 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:03.866303 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:03.866304 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:03.866305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:03.866453 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:03.866524 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:03.866774 (+ 250us) server_negotiation.cc:299] Negotiation successful
0809 02:15:03.866856 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:15:03.867010 17973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:03.860240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.65:40765)
0809 02:15:03.860438 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:03.860453 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:03.860527 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:03.860670 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:03.860671 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:03.860945 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:03.861038 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:03.861041 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.861744 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.861745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:03.862282 (+ 537us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:03.862284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:03.862656 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:03.862657 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:03.862710 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:03.863250 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:03.863256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:03.863596 (+ 340us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:03.865799 (+ 2203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:03.865800 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:03.865801 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:03.865978 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:03.866185 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:03.866186 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:03.866186 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:03.866242 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:03.866539 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:03.866540 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:03.866652 (+ 112us) client_negotiation.cc:769] Sending connection context
0809 02:15:03.866737 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:15:03.866867 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
I20260809 02:15:03.883832 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33203
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:03.969044 17975 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:03.969352 17975 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:03.969396 17975 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:03.970561 17975 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:03.970633 17975 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:03.970670 17975 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:03.970706 17975 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:03.972242 17975 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17975
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:03.972533 17975 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:03.972842 17975 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:03.978302 17981 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:03.978307 17980 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:03.978338 17975 server_base.cc:1034] running on GCE node
W20260809 02:15:03.978320 17983 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:03.978664 17975 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:03.978926 17975 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:03.980124 17975 hybrid_clock.cc:648] HybridClock initialized: now 1786241703980104 us; error 41 us; skew 500 ppm
Aug 09 02:15:03 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:03.983158 17975 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260809 02:15:03.983659 17975 webserver.cc:460] Webserver started at http://127.9.40.66:34687/ using document root <none> and password file <none>
I20260809 02:15:03.983915 17975 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:03.983994 17975 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:03.984158 17975 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:03.985337 17975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "5ab8f35d91c5449a83e4277788cb4b68"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "920d460dad8d1e4ca7e4e53bd1b56fbb"
server_key_iv: "edae5bf5dc9c257ea032b80a6d20adaf"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.985723 17975 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "5ab8f35d91c5449a83e4277788cb4b68"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "920d460dad8d1e4ca7e4e53bd1b56fbb"
server_key_iv: "edae5bf5dc9c257ea032b80a6d20adaf"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.987157 17975 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:15:03.987960 17990 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:03.988189 17975 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:03.988301 17975 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "5ab8f35d91c5449a83e4277788cb4b68"
format_stamp: "Formatted at 2026-08-09 02:15:03 on dist-test-slave-qg60"
server_key: "920d460dad8d1e4ca7e4e53bd1b56fbb"
server_key_iv: "edae5bf5dc9c257ea032b80a6d20adaf"
server_key_version: "encryptionkey@0"
I20260809 02:15:03.988380 17975 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:04.003661 17975 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:04.006361 17975 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:04.006575 17975 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:04.007416 17975 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:04.007478 17975 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.007541 17975 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:04.007572 17975 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:04.015681 17975 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:38973
I20260809 02:15:04.015798 18118 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:38973 every 8 connection(s)
I20260809 02:15:04.016103 17975 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:04.018631 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.008020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:41553 (local address 127.9.40.126:33203)
0809 02:15:04.008065 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.008068 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.008373 (+ 305us) server_negotiation.cc:407] Connection header received
0809 02:15:04.009057 (+ 684us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.009059 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.009072 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.009131 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.010035 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.010978 (+ 943us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.012335 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.012635 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.012746 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.015481 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.015483 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.015483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.015501 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.017247 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.017703 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.017704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.017704 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.017768 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.018069 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.018070 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.018070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.018188 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.018243 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.018470 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.018560 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:15:04.018751 18000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.008035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.66:41553)
0809 02:15:04.008241 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.008283 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.008942 (+ 659us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.009203 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.009211 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.009611 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:04.009906 (+ 295us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.009913 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.011120 (+ 1207us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.011121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.012237 (+ 1116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.012240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.012783 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.012784 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.012940 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.013575 (+ 635us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:04.013585 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:04.015341 (+ 1756us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:04.017346 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.017353 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.017354 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.017623 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.017884 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.017884 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.017885 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.018002 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.018233 (+ 231us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:04.018236 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:04.018347 (+ 111us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.018425 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.018556 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":107,"threads_started":1}
I20260809 02:15:04.019361 18119 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33203
I20260809 02:15:04.019488 18119 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:04.019737 18119 heartbeater.cc:511] Master 127.9.40.126:33203 requested a full tablet report, sending...
I20260809 02:15:04.020074 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 17975
I20260809 02:15:04.020222 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:04.020468 9377 external_mini_cluster.cc:1428] Setting key b8276c2787a734668dcecf11fb9f4591
I20260809 02:15:04.020887 17743 ts_manager.cc:194] Registered new tserver with Master: 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973)
I20260809 02:15:04.021379 17743 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:41553
I20260809 02:15:04.029547 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.020636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35699 (local address 127.9.40.126:33203)
0809 02:15:04.020789 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.020790 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.020803 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:15:04.020827 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.020828 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.020838 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.020884 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.023138 (+ 2254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.023809 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.024764 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.025120 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.025196 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.026222 (+ 1026us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.026224 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.026224 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.026233 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.028139 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.028595 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.028596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.028597 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.028657 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.028928 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.028928 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.028929 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.029117 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.029184 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.029387 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.029463 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124}
I20260809 02:15:04.029654 18125 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.020560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.66:35699)
0809 02:15:04.020684 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.020696 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.020778 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.020896 (+ 118us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.020897 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.021106 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:04.021194 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.021196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.023926 (+ 2730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.023927 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.024639 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.024642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.025215 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.025216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.025268 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.025770 (+ 502us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:04.025783 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:04.026126 (+ 343us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:04.028231 (+ 2105us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.028232 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.028233 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.028506 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.028765 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.028767 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.028767 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.028839 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.029196 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:04.029197 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:04.029270 (+ 73us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.029355 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.029497 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":97,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260809 02:15:04.051867 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33203
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:04.135375 18128 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:04.135593 18128 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:04.135618 18128 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:04.137023 18128 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:04.137079 18128 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:04.137100 18128 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260809 02:15:04.137118 18128 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:04.138926 18128 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.9.40.126:33203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18128
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:04.139180 18128 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:04.139426 18128 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:04.144455 18134 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:04.144433 18133 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:04.144445 18136 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:04.144655 18128 server_base.cc:1034] running on GCE node
I20260809 02:15:04.144939 18128 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:04.145310 18128 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:04.146493 18128 hybrid_clock.cc:648] HybridClock initialized: now 1786241704146471 us; error 33 us; skew 500 ppm
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:04.149457 18128 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260809 02:15:04.149961 18128 webserver.cc:460] Webserver started at http://127.9.40.67:38139/ using document root <none> and password file <none>
I20260809 02:15:04.150202 18128 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:04.150272 18128 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:04.150432 18128 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:04.151559 18128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "9ecd4b34a2254b4bae566d98461328c6"
format_stamp: "Formatted at 2026-08-09 02:15:04 on dist-test-slave-qg60"
server_key: "ae31aa9c1f00b7a45a161c40582c1020"
server_key_iv: "058a0d1163d81f295a139f65142e978d"
server_key_version: "encryptionkey@0"
I20260809 02:15:04.151940 18128 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "9ecd4b34a2254b4bae566d98461328c6"
format_stamp: "Formatted at 2026-08-09 02:15:04 on dist-test-slave-qg60"
server_key: "ae31aa9c1f00b7a45a161c40582c1020"
server_key_iv: "058a0d1163d81f295a139f65142e978d"
server_key_version: "encryptionkey@0"
I20260809 02:15:04.153437 18128 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:04.154309 18143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.154507 18128 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:04.154614 18128 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "9ecd4b34a2254b4bae566d98461328c6"
format_stamp: "Formatted at 2026-08-09 02:15:04 on dist-test-slave-qg60"
server_key: "ae31aa9c1f00b7a45a161c40582c1020"
server_key_iv: "058a0d1163d81f295a139f65142e978d"
server_key_version: "encryptionkey@0"
I20260809 02:15:04.154711 18128 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:04.182324 18128 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:04.184443 18128 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:04.184621 18128 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:04.185268 18128 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:04.185329 18128 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.185374 18128 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:04.185403 18128 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:04.192705 18128 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:46119
I20260809 02:15:04.192821 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:46119 every 8 connection(s)
I20260809 02:15:04.193161 18128 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:04.195410 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.185722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43301 (local address 127.9.40.126:33203)
0809 02:15:04.185783 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.185786 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.186035 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:15:04.186592 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.186594 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.186607 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.186666 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.187530 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.188410 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.189576 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.189787 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.189876 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.192761 (+ 2885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.192763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.192763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.192772 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.194278 (+ 1506us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.194661 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.194662 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.194663 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.194717 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.194906 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.194906 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.194906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.195022 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.195081 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.195280 (+ 199us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.195344 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:15:04.195582 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.185696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.67:43301)
0809 02:15:04.185858 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.185896 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.186483 (+ 587us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.186738 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.186742 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.187119 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:04.187418 (+ 299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.187424 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.188510 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.188511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.189461 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.189464 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.189905 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.189906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.190071 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.190615 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:04.190627 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:04.192604 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:04.194357 (+ 1753us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.194363 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.194364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.194590 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.194771 (+ 181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.194771 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.194772 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.194859 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.195070 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:04.195074 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:04.195188 (+ 114us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.195268 (+ 80us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.195389 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260809 02:15:04.196041 18266 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33203
I20260809 02:15:04.196123 18266 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:04.196317 18266 heartbeater.cc:511] Master 127.9.40.126:33203 requested a full tablet report, sending...
I20260809 02:15:04.196755 17744 ts_manager.cc:194] Registered new tserver with Master: 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.197124 17744 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43301
I20260809 02:15:04.198297 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18128
I20260809 02:15:04.198390 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:04.198594 9377 external_mini_cluster.cc:1428] Setting key 841b80b6352a9d8e703c366a72063a0a
I20260809 02:15:04.199524 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:15:04.203584 18274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.197114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.9.40.67:45007)
0809 02:15:04.197228 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.197239 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.197300 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.197465 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.197466 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.197674 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:04.197757 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.197759 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.198587 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.198588 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.199348 (+ 760us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.199351 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.199859 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.199860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.199934 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.200380 (+ 446us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:04.200384 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:04.200653 (+ 269us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:04.202417 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.202418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.202418 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.202718 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.202941 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.202942 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.202942 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.202999 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.203267 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:04.203268 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:04.203328 (+ 60us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.203410 (+ 82us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.203509 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":88,"thread_start_us":64,"threads_started":1}
I20260809 02:15:04.203661 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.197176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45007 (local address 127.9.40.126:33203)
0809 02:15:04.197218 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.197220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.197285 (+ 65us) server_negotiation.cc:407] Connection header received
0809 02:15:04.197351 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.197352 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.197361 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.197427 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.197835 (+ 408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.198463 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.199524 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.199781 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.199860 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.200771 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.200772 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.200773 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.200779 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.202300 (+ 1521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.202800 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.202802 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.202802 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.202864 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.203065 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.203066 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.203066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.203195 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.203250 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.203443 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.203529 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:15:04.224490 17819 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:04.218280 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59282 (local address 127.9.40.126:33203)
0809 02:15:04.218343 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.218346 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.218356 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:04.218450 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.218451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.218465 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.218538 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.219186 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.219775 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.220570 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.220842 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.220918 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.224370 (+ 3452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59282: BlockingRecv error: recv got EOF from 127.0.0.1:59282 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:04.232913 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.225387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59284 (local address 127.9.40.126:33203)
0809 02:15:04.225440 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.225442 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.225463 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:15:04.225528 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.225529 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.225537 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.225592 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.226214 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.226698 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.227448 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.227631 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.227696 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.230208 (+ 2512us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.230209 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.230210 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.230217 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.231669 (+ 1452us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.232081 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.232082 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.232083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.232139 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.232339 (+ 200us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.232340 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.232340 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.232451 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.232536 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.232700 (+ 164us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.232832 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:15:04.234858 17744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:04.236538 17744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:04.243835 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51088 (local address 127.9.40.66:38973)
0809 02:15:04.240072 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.240078 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.240090 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:15:04.240132 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.240134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.240149 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.240253 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.240471 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.241320 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.242707 (+ 1387us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.243374 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.243469 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.243561 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.243695 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.243735 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":89,"threads_started":1}
I20260809 02:15:04.243818 18290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.0.0.1:51088)
0809 02:15:04.239817 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.239829 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.239882 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.240282 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.240285 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.240287 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.240362 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.240364 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.241420 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.241421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.242614 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.242617 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.243468 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.243469 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.243580 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.243594 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.243669 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.243724 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":88,"threads_started":1}
I20260809 02:15:04.244204 18291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39083 (local address 127.0.0.1:53404)
0809 02:15:04.239725 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.239771 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.239843 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.240430 (+ 587us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.240431 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.240433 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.240480 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.240481 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.241898 (+ 1417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.241899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.242999 (+ 1100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.243001 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.243963 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.243964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.244058 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.244061 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.244091 (+ 30us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.244128 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":74,"threads_started":1}
I20260809 02:15:04.244602 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.9.40.65:39083)
0809 02:15:04.240244 (+ 616us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.240249 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.240261 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:15:04.240306 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.240308 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.240323 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.240421 (+ 98us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.240558 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.241773 (+ 1215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.243112 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.243857 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.244328 (+ 471us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.244428 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.244454 (+ 26us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.244498 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":101,"threads_started":1}
I20260809 02:15:04.244688 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 269637dc010e4177b6ca28232e24b163 (DEFAULT_TABLE table=test-table [id=36547cd319fd40caba186fb89262a5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.245167 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269637dc010e4177b6ca28232e24b163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.245272 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 269637dc010e4177b6ca28232e24b163 (DEFAULT_TABLE table=test-table [id=36547cd319fd40caba186fb89262a5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.245596 18295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46119 (local address 127.0.0.1:48944)
0809 02:15:04.240014 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.240034 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.240081 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.240576 (+ 495us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.240577 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.240578 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.240606 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.240607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.242781 (+ 2174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.242782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.243849 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.243853 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.245362 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.245363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.245431 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.245435 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.245485 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.245536 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":79,"threads_started":1}
I20260809 02:15:04.245610 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269637dc010e4177b6ca28232e24b163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.245919 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.239972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48944 (local address 127.9.40.67:46119)
0809 02:15:04.240274 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.240278 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.240287 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:04.240324 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.240326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.240340 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.240420 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.240682 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.241773 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.244392 (+ 2619us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.245177 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.245245 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.245326 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.245785 (+ 459us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.245829 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":270,"thread_start_us":130,"threads_started":1}
I20260809 02:15:04.246474 18195 tablet_service.cc:1467] Processing CreateTablet for tablet 269637dc010e4177b6ca28232e24b163 (DEFAULT_TABLE table=test-table [id=36547cd319fd40caba186fb89262a5ee]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.246799 18195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 269637dc010e4177b6ca28232e24b163. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.249405 18298 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Bootstrap starting.
I20260809 02:15:04.249838 18299 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Bootstrap starting.
I20260809 02:15:04.250181 18298 tablet_bootstrap.cc:654] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.250458 18299 tablet_bootstrap.cc:654] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.250494 18298 log.cc:824] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:04.250815 18300 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Bootstrap starting.
I20260809 02:15:04.250847 18299 log.cc:824] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:04.251462 18298 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: No bootstrap required, opened a new log
I20260809 02:15:04.251503 18300 tablet_bootstrap.cc:654] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.251538 18298 ts_tablet_manager.cc:1397] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:15:04.251586 18299 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: No bootstrap required, opened a new log
I20260809 02:15:04.251652 18299 ts_tablet_manager.cc:1397] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:04.251813 18300 log.cc:824] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:04.252633 18300 tablet_bootstrap.cc:492] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: No bootstrap required, opened a new log
I20260809 02:15:04.252708 18300 ts_tablet_manager.cc:1397] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:04.253329 18299 raft_consensus.cc:348] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.253494 18299 raft_consensus.cc:374] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.253537 18299 raft_consensus.cc:729] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.253661 18299 consensus_queue.cc:260] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.253726 18298 raft_consensus.cc:348] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.253886 18298 raft_consensus.cc:374] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.253938 18298 raft_consensus.cc:729] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.253986 18299 ts_tablet_manager.cc:1428] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:15:04.254141 17966 heartbeater.cc:503] Master 127.9.40.126:33203 was elected leader, sending a full tablet report...
I20260809 02:15:04.254078 18298 consensus_queue.cc:260] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.254384 18298 ts_tablet_manager.cc:1428] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:15:04.254420 18119 heartbeater.cc:503] Master 127.9.40.126:33203 was elected leader, sending a full tablet report...
I20260809 02:15:04.254774 18300 raft_consensus.cc:348] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.254945 18300 raft_consensus.cc:374] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.255009 18300 raft_consensus.cc:729] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.255122 18300 consensus_queue.cc:260] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.255393 18300 ts_tablet_manager.cc:1428] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:15:04.255415 18266 heartbeater.cc:503] Master 127.9.40.126:33203 was elected leader, sending a full tablet report...
W20260809 02:15:04.267041 18120 tablet.cc:2369] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:04.352662 17967 tablet.cc:2369] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:04.443894 18268 tablet.cc:2369] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:04.501214 18306 raft_consensus.cc:482] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:04.501361 18306 raft_consensus.cc:504] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.501693 18304 raft_consensus.cc:482] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:04.501804 18306 leader_election.cc:288] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973)
I20260809 02:15:04.501844 18304 raft_consensus.cc:504] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.502317 18304 leader_election.cc:288] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.505628 18152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39083 (local address 127.9.40.67:52699)
0809 02:15:04.502112 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.502125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.502181 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.502418 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.502420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.502422 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.502523 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.502526 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.503460 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.503461 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.504617 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.504620 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.505362 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.505363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.505438 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.505442 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.505501 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.505553 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 02:15:04.506162 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:52699 (local address 127.9.40.65:39083)
0809 02:15:04.502239 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.502241 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.502250 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:04.502279 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.502280 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.502294 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.502356 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.502618 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.503374 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.504748 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.505258 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.505323 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.505332 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.506047 (+ 715us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.506086 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":21}
I20260809 02:15:04.506476 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35247 (local address 127.9.40.67:46119)
0809 02:15:04.502804 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.502806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.503101 (+ 295us) server_negotiation.cc:407] Connection header received
0809 02:15:04.503209 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.503209 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.503218 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.503245 (+ 27us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.503433 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.504111 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.505587 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506109 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506152 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506159 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.506384 (+ 225us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.506417 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:15:04.506433 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9" is_pre_election: true
I20260809 02:15:04.506558 17917 raft_consensus.cc:2454] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 0.
I20260809 02:15:04.506747 18308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46119 (local address 127.9.40.65:35247)
0809 02:15:04.502976 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.502986 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.503142 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.503295 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.503296 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.503297 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.503358 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.503359 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.504208 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.504209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.505498 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.505501 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506217 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506218 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.506297 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506300 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.506590 (+ 290us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.506634 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"mutex_wait_us":91,"thread_start_us":55,"threads_started":1}
I20260809 02:15:04.506891 18294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:52849 (local address 127.9.40.66:38973)
0809 02:15:04.502139 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.502141 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.502352 (+ 211us) server_negotiation.cc:407] Connection header received
0809 02:15:04.502504 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.502505 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.502520 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.502555 (+ 35us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.502837 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.503648 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.505720 (+ 2072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506407 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506495 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506505 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.506737 (+ 232us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.506788 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":27}
I20260809 02:15:04.506904 18144 leader_election.cc:302] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ecd4b34a2254b4bae566d98461328c6, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.506899 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53135 (local address 127.9.40.66:38973)
0809 02:15:04.502781 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.502783 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.502943 (+ 160us) server_negotiation.cc:407] Connection header received
0809 02:15:04.503042 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.503043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.503052 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.503102 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.503308 (+ 206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.504080 (+ 772us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.505729 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506419 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506496 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506506 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.506737 (+ 231us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.506790 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":88,"thread_start_us":59,"threads_started":1}
I20260809 02:15:04.507084 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6" is_pre_election: true
I20260809 02:15:04.507148 18309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.9.40.65:53135)
0809 02:15:04.502865 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.502876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.502954 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.503164 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.503165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.503167 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.503231 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.503233 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.504192 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.504193 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.505630 (+ 1437us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.505633 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506545 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506546 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.506639 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506643 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.506809 (+ 166us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.507038 (+ 229us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":73,"threads_started":1}
I20260809 02:15:04.507045 18306 raft_consensus.cc:2781] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:04.507274 18307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.502034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.9.40.67:52849)
0809 02:15:04.502287 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.502298 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.502383 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.502641 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.502642 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.502644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.502734 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.502737 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.503749 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.503749 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.505593 (+ 1844us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.505597 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.506564 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.506565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.506639 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.506643 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.506994 (+ 351us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.507039 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":236,"mutex_wait_us":75,"thread_start_us":108,"threads_started":1}
I20260809 02:15:04.507304 18306 raft_consensus.cc:482] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:04.507455 18306 raft_consensus.cc:3037] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.507627 18069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68" is_pre_election: true
I20260809 02:15:04.507804 18069 raft_consensus.cc:2454] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 0.
I20260809 02:15:04.507874 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68" is_pre_election: true
I20260809 02:15:04.507965 18067 raft_consensus.cc:2454] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 0.
I20260809 02:15:04.508057 17841 leader_election.cc:302] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.508210 18304 raft_consensus.cc:2781] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:04.508272 18304 raft_consensus.cc:482] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:04.508289 18304 raft_consensus.cc:3037] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.508505 18306 raft_consensus.cc:504] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.508659 18306 leader_election.cc:288] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 election: Requested vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973)
I20260809 02:15:04.508682 18218 raft_consensus.cc:2379] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd5136614a8d4f9082c100c0525324e9 in current term 1: Already voted for candidate 9ecd4b34a2254b4bae566d98461328c6 in this term.
I20260809 02:15:04.508945 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68"
I20260809 02:15:04.508926 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9"
I20260809 02:15:04.509004 18067 raft_consensus.cc:3037] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.509444 18304 raft_consensus.cc:504] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.509617 17917 raft_consensus.cc:2379] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate 9ecd4b34a2254b4bae566d98461328c6 in current term 1: Already voted for candidate bd5136614a8d4f9082c100c0525324e9 in this term.
I20260809 02:15:04.509652 18304 leader_election.cc:288] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.509783 18069 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68"
I20260809 02:15:04.509840 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "269637dc010e4177b6ca28232e24b163" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6"
I20260809 02:15:04.509914 18218 raft_consensus.cc:2379] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election vote request: Denying vote to candidate bd5136614a8d4f9082c100c0525324e9 in current term 1: Already voted for candidate 9ecd4b34a2254b4bae566d98461328c6 in this term.
I20260809 02:15:04.510107 17841 leader_election.cc:302] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd5136614a8d4f9082c100c0525324e9; no voters: 5ab8f35d91c5449a83e4277788cb4b68, 9ecd4b34a2254b4bae566d98461328c6
I20260809 02:15:04.510190 18067 raft_consensus.cc:2454] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 1.
I20260809 02:15:04.510217 18304 raft_consensus.cc:2726] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election lost for term 1. Reason: could not achieve majority
I20260809 02:15:04.510350 18145 leader_election.cc:302] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, 9ecd4b34a2254b4bae566d98461328c6; no voters: bd5136614a8d4f9082c100c0525324e9
I20260809 02:15:04.510473 18306 raft_consensus.cc:2781] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:04.510686 18306 raft_consensus.cc:686] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Running, Role: LEADER
I20260809 02:15:04.510792 18306 consensus_queue.cc:237] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.511761 17744 catalog_manager.cc:5495] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "9ecd4b34a2254b4bae566d98461328c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } health_report { overall_health: HEALTHY } } }
I20260809 02:15:04.527491 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.525053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48960 (local address 127.9.40.67:46119)
0809 02:15:04.525120 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.525121 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.525128 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:15:04.525162 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.525169 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.525186 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.525266 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:04.525423 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.525922 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.526700 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.526897 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.526971 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.527148 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.527255 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.527341 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.527382 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.527404 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:15:04.529394 17917 raft_consensus.cc:1261] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Refusing update from remote peer 9ecd4b34a2254b4bae566d98461328c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.529613 18067 raft_consensus.cc:1261] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Refusing update from remote peer 9ecd4b34a2254b4bae566d98461328c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.529661 18311 consensus_queue.cc:1035] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.529901 18311 consensus_queue.cc:1035] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.532347 18314 mvcc.cc:205] Tried to move back new op lower bound from 7316446021749334016 to 7316446021676748800. Current Snapshot: MvccSnapshot[applied={T|T < 7316446021749334016}]
I20260809 02:15:04.532615 18316 mvcc.cc:205] Tried to move back new op lower bound from 7316446021749334016 to 7316446021676748800. Current Snapshot: MvccSnapshot[applied={T|T < 7316446021749334016}]
I20260809 02:15:04.533792 18315 mvcc.cc:205] Tried to move back new op lower bound from 7316446021749334016 to 7316446021676748800. Current Snapshot: MvccSnapshot[applied={T|T < 7316446021749334016 or (T in {7316446021749334016})}]
I20260809 02:15:04.535233 17744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.535326 17744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.536130 17744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Sending DeleteTablet for 3 replicas of tablet 269637dc010e4177b6ca28232e24b163
I20260809 02:15:04.536469 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 269637dc010e4177b6ca28232e24b163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53404
I20260809 02:15:04.536517 18195 tablet_service.cc:1514] Processing DeleteTablet for tablet 269637dc010e4177b6ca28232e24b163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48944
I20260809 02:15:04.536674 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 269637dc010e4177b6ca28232e24b163 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51088
I20260809 02:15:04.536928 18327 tablet_replica.cc:330] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: stopping tablet replica
I20260809 02:15:04.537114 18327 raft_consensus.cc:2229] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.537206 17744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:15:04.537317 18327 raft_consensus.cc:2258] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:15:04.537343 17744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:04.538136 18327 ts_tablet_manager.cc:1905] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.539203 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 80370da7a35d4c85b858b2bc336942a2 (DEFAULT_TABLE table=test-table [id=215759b0dcb94435beb23944c98c6638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.539273 18195 tablet_service.cc:1467] Processing CreateTablet for tablet 80370da7a35d4c85b858b2bc336942a2 (DEFAULT_TABLE table=test-table [id=215759b0dcb94435beb23944c98c6638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.539351 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80370da7a35d4c85b858b2bc336942a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.539389 18195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80370da7a35d4c85b858b2bc336942a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.540146 18327 ts_tablet_manager.cc:1918] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.540222 18327 log.cc:1196] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/269637dc010e4177b6ca28232e24b163
I20260809 02:15:04.540478 18327 ts_tablet_manager.cc:1939] T 269637dc010e4177b6ca28232e24b163 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting consensus metadata
I20260809 02:15:04.540779 18299 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Bootstrap starting.
I20260809 02:15:04.540966 18300 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Bootstrap starting.
I20260809 02:15:04.541280 17729 catalog_manager.cc:4841] TS 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973): tablet 269637dc010e4177b6ca28232e24b163 (table test-table [id=36547cd319fd40caba186fb89262a5ee]) successfully deleted
I20260809 02:15:04.541429 18300 tablet_bootstrap.cc:654] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.541471 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 80370da7a35d4c85b858b2bc336942a2 (DEFAULT_TABLE table=test-table [id=215759b0dcb94435beb23944c98c6638]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.541513 18299 tablet_bootstrap.cc:654] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.541577 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 80370da7a35d4c85b858b2bc336942a2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.542268 18299 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: No bootstrap required, opened a new log
I20260809 02:15:04.542317 18299 ts_tablet_manager.cc:1397] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:15:04.542464 18299 raft_consensus.cc:348] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.542533 18299 raft_consensus.cc:374] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.542552 18299 raft_consensus.cc:729] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.542622 18299 consensus_queue.cc:260] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.542651 18298 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Bootstrap starting.
I20260809 02:15:04.542706 18299 ts_tablet_manager.cc:1428] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.543076 18298 tablet_bootstrap.cc:654] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.543359 18325 tablet_replica.cc:330] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: stopping tablet replica
I20260809 02:15:04.543555 18325 raft_consensus.cc:2229] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.543735 18325 raft_consensus.cc:2258] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.543795 18298 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: No bootstrap required, opened a new log
I20260809 02:15:04.543844 18298 ts_tablet_manager.cc:1397] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:04.544011 18298 raft_consensus.cc:348] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.544091 18298 raft_consensus.cc:374] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.544119 18298 raft_consensus.cc:729] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.544168 18298 consensus_queue.cc:260] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.544252 18298 ts_tablet_manager.cc:1428] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.544272 18325 ts_tablet_manager.cc:1905] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.545780 18326 tablet_replica.cc:330] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: stopping tablet replica
I20260809 02:15:04.545909 18326 raft_consensus.cc:2229] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:04.546041 18326 raft_consensus.cc:2258] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.546083 18325 ts_tablet_manager.cc:1918] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.546190 18325 log.cc:1196] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/269637dc010e4177b6ca28232e24b163
I20260809 02:15:04.546497 18325 ts_tablet_manager.cc:1939] T 269637dc010e4177b6ca28232e24b163 P bd5136614a8d4f9082c100c0525324e9: Deleting consensus metadata
I20260809 02:15:04.546602 18326 ts_tablet_manager.cc:1905] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.546945 17728 catalog_manager.cc:4841] TS bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083): tablet 269637dc010e4177b6ca28232e24b163 (table test-table [id=36547cd319fd40caba186fb89262a5ee]) successfully deleted
I20260809 02:15:04.548311 18300 tablet_bootstrap.cc:492] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: No bootstrap required, opened a new log
I20260809 02:15:04.548355 18326 ts_tablet_manager.cc:1918] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.548372 18300 ts_tablet_manager.cc:1397] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.001s
I20260809 02:15:04.548411 18326 log.cc:1196] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/269637dc010e4177b6ca28232e24b163
I20260809 02:15:04.548534 18300 raft_consensus.cc:348] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.548628 18300 raft_consensus.cc:374] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.548656 18300 raft_consensus.cc:729] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.548712 18300 consensus_queue.cc:260] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.548806 18326 ts_tablet_manager.cc:1939] T 269637dc010e4177b6ca28232e24b163 P 9ecd4b34a2254b4bae566d98461328c6: Deleting consensus metadata
I20260809 02:15:04.548846 18300 ts_tablet_manager.cc:1428] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:15:04.549193 17728 catalog_manager.cc:4841] TS 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119): tablet 269637dc010e4177b6ca28232e24b163 (table test-table [id=36547cd319fd40caba186fb89262a5ee]) successfully deleted
I20260809 02:15:04.584205 18311 raft_consensus.cc:482] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:04.584339 18311 raft_consensus.cc:504] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.584554 18311 leader_election.cc:288] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973)
I20260809 02:15:04.584831 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80370da7a35d4c85b858b2bc336942a2" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9" is_pre_election: true
I20260809 02:15:04.584834 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80370da7a35d4c85b858b2bc336942a2" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68" is_pre_election: true
I20260809 02:15:04.584952 17917 raft_consensus.cc:2454] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 0.
I20260809 02:15:04.584981 18067 raft_consensus.cc:2454] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 0.
I20260809 02:15:04.585172 18144 leader_election.cc:302] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ecd4b34a2254b4bae566d98461328c6, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.585284 18311 raft_consensus.cc:2781] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:04.585333 18311 raft_consensus.cc:482] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:04.585350 18311 raft_consensus.cc:3037] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.586133 18311 raft_consensus.cc:504] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.586294 18311 leader_election.cc:288] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 election: Requested vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973)
I20260809 02:15:04.586436 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80370da7a35d4c85b858b2bc336942a2" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9"
I20260809 02:15:04.586555 17917 raft_consensus.cc:3037] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.586536 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "80370da7a35d4c85b858b2bc336942a2" candidate_uuid: "9ecd4b34a2254b4bae566d98461328c6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68"
I20260809 02:15:04.586614 18067 raft_consensus.cc:3037] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.587414 18067 raft_consensus.cc:2454] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 1.
I20260809 02:15:04.587527 17917 raft_consensus.cc:2454] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9ecd4b34a2254b4bae566d98461328c6 in term 1.
I20260809 02:15:04.587656 18145 leader_election.cc:302] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, 9ecd4b34a2254b4bae566d98461328c6; no voters:
I20260809 02:15:04.587772 18311 raft_consensus.cc:2781] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:04.587860 18311 raft_consensus.cc:686] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 LEADER]: Becoming Leader. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Running, Role: LEADER
I20260809 02:15:04.587913 18311 consensus_queue.cc:237] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.588598 17743 catalog_manager.cc:5495] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "9ecd4b34a2254b4bae566d98461328c6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } health_report { overall_health: HEALTHY } } }
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241703, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.126@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:04.633023 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.625698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59290 (local address 127.9.40.126:33203)
0809 02:15:04.625773 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.625776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.625930 (+ 154us) server_negotiation.cc:407] Connection header received
0809 02:15:04.625995 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.625996 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.626006 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.626053 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.626546 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.627044 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.627769 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.627993 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.628056 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.630308 (+ 2252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.630310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.630310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.630318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.631780 (+ 1462us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.632183 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.632184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.632185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.632254 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.632466 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.632467 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.632468 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.632582 (+ 114us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.632653 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.632827 (+ 174us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.632916 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:15:04.633116 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.625632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33203 (local address 127.0.0.1:59290)
0809 02:15:04.625833 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.625851 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.625937 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.626066 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.626066 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.626388 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:04.626442 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.626444 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.627136 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.627138 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.627679 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.627681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.628065 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.628065 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.628120 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.628706 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:04.628711 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:04.630211 (+ 1500us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:04.631883 (+ 1672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.631892 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.631894 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.632104 (+ 210us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.632332 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:04.632333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:04.632333 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:04.632391 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:04.632656 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:04.632664 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:04.632708 (+ 44us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.632804 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.632992 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":180,"mutex_wait_us":83,"thread_start_us":107,"threads_started":1}
I20260809 02:15:04.635483 17743 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59290:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260809 02:15:04.635710 17743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:04.637535 18047 tablet_service.cc:1467] Processing CreateTablet for tablet dc84b61967614a9ca9473e1e80d7aad3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f718e22f967d41b4b8a104d068bf8388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:04.637526 18195 tablet_service.cc:1467] Processing CreateTablet for tablet dc84b61967614a9ca9473e1e80d7aad3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f718e22f967d41b4b8a104d068bf8388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:04.637571 17895 tablet_service.cc:1467] Processing CreateTablet for tablet dc84b61967614a9ca9473e1e80d7aad3 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=f718e22f967d41b4b8a104d068bf8388]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260809 02:15:04.637701 18195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc84b61967614a9ca9473e1e80d7aad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.637701 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc84b61967614a9ca9473e1e80d7aad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.637737 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dc84b61967614a9ca9473e1e80d7aad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.639251 18300 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: Bootstrap starting.
I20260809 02:15:04.639251 18298 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: Bootstrap starting.
I20260809 02:15:04.639429 18299 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: Bootstrap starting.
I20260809 02:15:04.639726 18298 tablet_bootstrap.cc:654] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.639725 18300 tablet_bootstrap.cc:654] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.639899 18299 tablet_bootstrap.cc:654] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.640851 18299 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: No bootstrap required, opened a new log
I20260809 02:15:04.640903 18299 ts_tablet_manager.cc:1397] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:04.640918 18298 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: No bootstrap required, opened a new log
I20260809 02:15:04.640962 18298 ts_tablet_manager.cc:1397] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:04.641098 18299 raft_consensus.cc:348] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641109 18298 raft_consensus.cc:348] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641151 18300 tablet_bootstrap.cc:492] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: No bootstrap required, opened a new log
I20260809 02:15:04.641177 18299 raft_consensus.cc:374] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.641182 18298 raft_consensus.cc:374] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.641204 18299 raft_consensus.cc:729] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.641206 18298 raft_consensus.cc:729] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.641211 18300 ts_tablet_manager.cc:1397] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:15:04.641250 18299 consensus_queue.cc:260] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641250 18298 consensus_queue.cc:260] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641340 18299 ts_tablet_manager.cc:1428] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.641340 18298 ts_tablet_manager.cc:1428] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.641394 18300 raft_consensus.cc:348] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641474 18300 raft_consensus.cc:374] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.641453 18304 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.641503 18300 raft_consensus.cc:729] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.641517 18304 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.641563 18300 consensus_queue.cc:260] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.641675 18300 ts_tablet_manager.cc:1428] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.641692 18311 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.641794 18311 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.641841 18305 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.641932 18305 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.678709 18305 raft_consensus.cc:482] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:04.678817 18305 raft_consensus.cc:504] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.679195 18305 leader_election.cc:288] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.682523 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.679477 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42173 (local address 127.9.40.65:39083)
0809 02:15:04.679539 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.679541 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.679719 (+ 178us) server_negotiation.cc:407] Connection header received
0809 02:15:04.679756 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.679758 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.679771 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.679836 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.680054 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.680649 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.681754 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.682140 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.682193 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.682201 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.682403 (+ 202us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.682441 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:15:04.682526 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.679781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:36045 (local address 127.9.40.67:46119)
0809 02:15:04.679861 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.679864 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.679874 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:04.679957 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.679958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.679971 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.680108 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:04.680276 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.680824 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.681810 (+ 986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.682155 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.682245 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.682256 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.682408 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.682455 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:15:04.682538 18344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.679647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46119 (local address 127.9.40.66:36045)
0809 02:15:04.679816 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.679827 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.679887 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.680115 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.680116 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.680133 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.680200 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.680201 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.680916 (+ 715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.680916 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.681691 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.681694 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.682264 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.682265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.682324 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.682327 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.682393 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.682433 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":82,"threads_started":1}
I20260809 02:15:04.682909 18343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.679391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39083 (local address 127.9.40.66:42173)
0809 02:15:04.679574 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.679586 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.679658 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.679878 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.679879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.679882 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:04.679971 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.679974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.680769 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.680770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.681660 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.681662 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.682221 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.682223 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.682311 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.682315 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.682398 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.682444 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161,"thread_start_us":109,"threads_started":1}
I20260809 02:15:04.682924 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc84b61967614a9ca9473e1e80d7aad3" candidate_uuid: "5ab8f35d91c5449a83e4277788cb4b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6" is_pre_election: true
I20260809 02:15:04.683045 18218 raft_consensus.cc:2454] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab8f35d91c5449a83e4277788cb4b68 in term 0.
I20260809 02:15:04.683233 17991 leader_election.cc:302] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, 9ecd4b34a2254b4bae566d98461328c6; no voters:
I20260809 02:15:04.683290 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc84b61967614a9ca9473e1e80d7aad3" candidate_uuid: "5ab8f35d91c5449a83e4277788cb4b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9" is_pre_election: true
I20260809 02:15:04.683373 17917 raft_consensus.cc:2454] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5ab8f35d91c5449a83e4277788cb4b68 in term 0.
I20260809 02:15:04.683473 18305 raft_consensus.cc:2781] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:04.683534 18305 raft_consensus.cc:482] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:04.683558 18305 raft_consensus.cc:3037] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.684329 18305 raft_consensus.cc:504] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.684463 18305 leader_election.cc:288] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [CANDIDATE]: Term 1 election: Requested vote from peers bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.684811 17917 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc84b61967614a9ca9473e1e80d7aad3" candidate_uuid: "5ab8f35d91c5449a83e4277788cb4b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd5136614a8d4f9082c100c0525324e9"
I20260809 02:15:04.684835 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dc84b61967614a9ca9473e1e80d7aad3" candidate_uuid: "5ab8f35d91c5449a83e4277788cb4b68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6"
I20260809 02:15:04.684897 18218 raft_consensus.cc:3037] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.684897 17917 raft_consensus.cc:3037] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.685671 17917 raft_consensus.cc:2454] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab8f35d91c5449a83e4277788cb4b68 in term 1.
I20260809 02:15:04.685824 18218 raft_consensus.cc:2454] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5ab8f35d91c5449a83e4277788cb4b68 in term 1.
I20260809 02:15:04.685853 17991 leader_election.cc:302] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.685927 18305 raft_consensus.cc:2781] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:04.686136 18305 raft_consensus.cc:686] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 LEADER]: Becoming Leader. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Running, Role: LEADER
I20260809 02:15:04.686209 18305 consensus_queue.cc:237] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.686589 18346 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: TxnStatusTablet state changed. Reason: New leader 5ab8f35d91c5449a83e4277788cb4b68. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.686684 18346 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:04.686896 18348 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260809 02:15:04.687031 18348 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260809 02:15:04.686961 17744 catalog_manager.cc:5495] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:04.702433 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.699905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.9.40.66:38973)
0809 02:15:04.699964 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.699966 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.699974 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:04.700019 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.700019 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.700035 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.700111 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:04.700244 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.700791 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.701539 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.701747 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.701822 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.702065 (+ 243us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.702158 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.702254 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.702335 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.702354 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:15:04.702497 18339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.699686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.0.0.1:51100)
0809 02:15:04.699765 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.699777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.699828 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.700119 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.700120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.700122 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:04.700173 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.700176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.700860 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.700861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.701436 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.701437 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.701820 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.701821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.701880 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.701899 (+ 19us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.702257 (+ 358us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.702269 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.702334 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.702390 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:15:04.703770 17917 raft_consensus.cc:1261] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Refusing update from remote peer 5ab8f35d91c5449a83e4277788cb4b68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.703902 18218 raft_consensus.cc:1261] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Refusing update from remote peer 5ab8f35d91c5449a83e4277788cb4b68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.704071 18346 consensus_queue.cc:1035] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.704182 18305 consensus_queue.cc:1035] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.704804 18304 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: TxnStatusTablet state changed. Reason: New leader 5ab8f35d91c5449a83e4277788cb4b68. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.704907 18304 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.705253 18311 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: TxnStatusTablet state changed. Reason: New leader 5ab8f35d91c5449a83e4277788cb4b68. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.705339 18311 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.705753 18305 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.705840 18305 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:04.705941 18305 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.706017 18305 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 5ab8f35d91c5449a83e4277788cb4b68: This TxnStatusTablet replica's current role is: LEADER
I20260809 02:15:04.706172 18304 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.706264 18304 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P bd5136614a8d4f9082c100c0525324e9: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.706864 18311 tablet_replica.cc:439] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "5ab8f35d91c5449a83e4277788cb4b68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } } }
I20260809 02:15:04.706945 18311 tablet_replica.cc:442] T dc84b61967614a9ca9473e1e80d7aad3 P 9ecd4b34a2254b4bae566d98461328c6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260809 02:15:04.712267 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.709817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:55885 (local address 127.9.40.66:38973)
0809 02:15:04.709879 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.709881 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.710025 (+ 144us) server_negotiation.cc:407] Connection header received
0809 02:15:04.710091 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.710092 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.710103 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.710158 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:04.710287 (+ 129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.710756 (+ 469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.711566 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.711752 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.711838 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.712007 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.712044 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.712106 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.712167 (+ 61us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.712188 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:15:04.712283 18358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.709758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.9.40.67:55885)
0809 02:15:04.709953 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.709963 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.710042 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.710161 (+ 119us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.710162 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.710164 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:04.710213 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.710215 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.710848 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.710849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.711429 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.711431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.711842 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.711843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.711904 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.711922 (+ 18us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.712107 (+ 185us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.712110 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.712163 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.712212 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":82,"threads_started":1}
I20260809 02:15:04.714330 17917 raft_consensus.cc:1261] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Refusing update from remote peer 9ecd4b34a2254b4bae566d98461328c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.714339 18067 raft_consensus.cc:1261] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Refusing update from remote peer 9ecd4b34a2254b4bae566d98461328c6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.714550 18306 consensus_queue.cc:1035] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.714658 18311 consensus_queue.cc:1035] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.723235 18297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.720540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:37191 (local address 127.9.40.67:46119)
0809 02:15:04.720581 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.720583 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.720593 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:04.720657 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.720658 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.720666 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.720705 (+ 39us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:04.720880 (+ 175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.721332 (+ 452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.722533 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.722720 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.722776 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.722960 (+ 184us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.722985 (+ 25us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.723085 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.723164 (+ 79us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.723183 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:15:04.723510 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.720309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:46119 (local address 127.9.40.66:37191)
0809 02:15:04.720538 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:04.720550 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:04.720604 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:04.720717 (+ 113us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:04.720718 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:04.720719 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:04.720828 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.720830 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.721425 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.721426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.722444 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:04.722447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.722795 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.722797 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:04.722863 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.722880 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.723106 (+ 226us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.723108 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:15:04.723162 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:15:04.723207 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":127,"threads_started":1}
I20260809 02:15:04.732342 17743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.732472 17743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:59284:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.733242 17743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Sending DeleteTablet for 3 replicas of tablet 80370da7a35d4c85b858b2bc336942a2
I20260809 02:15:04.733578 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 80370da7a35d4c85b858b2bc336942a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53404
I20260809 02:15:04.733603 18195 tablet_service.cc:1514] Processing DeleteTablet for tablet 80370da7a35d4c85b858b2bc336942a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48944
I20260809 02:15:04.733695 18325 tablet_replica.cc:330] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: stopping tablet replica
I20260809 02:15:04.733706 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 80370da7a35d4c85b858b2bc336942a2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51088
I20260809 02:15:04.733760 18325 raft_consensus.cc:2229] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.733810 18325 raft_consensus.cc:2258] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.733804 18326 tablet_replica.cc:330] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: stopping tablet replica
I20260809 02:15:04.733868 18326 raft_consensus.cc:2229] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:04.733889 18327 tablet_replica.cc:330] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: stopping tablet replica
I20260809 02:15:04.733943 18326 raft_consensus.cc:2258] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.733940 18327 raft_consensus.cc:2229] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.733986 18327 raft_consensus.cc:2258] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.734320 18326 ts_tablet_manager.cc:1905] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.734323 18327 ts_tablet_manager.cc:1905] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.734551 18325 ts_tablet_manager.cc:1905] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.736351 18327 ts_tablet_manager.cc:1918] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:04.736366 18326 ts_tablet_manager.cc:1918] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:04.736433 18327 log.cc:1196] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/80370da7a35d4c85b858b2bc336942a2
I20260809 02:15:04.736434 18326 log.cc:1196] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/80370da7a35d4c85b858b2bc336942a2
I20260809 02:15:04.736680 18327 ts_tablet_manager.cc:1939] T 80370da7a35d4c85b858b2bc336942a2 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting consensus metadata
I20260809 02:15:04.736747 18326 ts_tablet_manager.cc:1939] T 80370da7a35d4c85b858b2bc336942a2 P 9ecd4b34a2254b4bae566d98461328c6: Deleting consensus metadata
I20260809 02:15:04.737115 18325 ts_tablet_manager.cc:1918] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:04.737112 17729 catalog_manager.cc:4841] TS 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973): tablet 80370da7a35d4c85b858b2bc336942a2 (table test-table [id=215759b0dcb94435beb23944c98c6638]) successfully deleted
I20260809 02:15:04.737112 17728 catalog_manager.cc:4841] TS 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119): tablet 80370da7a35d4c85b858b2bc336942a2 (table test-table [id=215759b0dcb94435beb23944c98c6638]) successfully deleted
I20260809 02:15:04.737187 18325 log.cc:1196] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/80370da7a35d4c85b858b2bc336942a2
I20260809 02:15:04.737407 18325 ts_tablet_manager.cc:1939] T 80370da7a35d4c85b858b2bc336942a2 P bd5136614a8d4f9082c100c0525324e9: Deleting consensus metadata
I20260809 02:15:04.737744 17728 catalog_manager.cc:4841] TS bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083): tablet 80370da7a35d4c85b858b2bc336942a2 (table test-table [id=215759b0dcb94435beb23944c98c6638]) successfully deleted
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:15:04.765758 17819 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:04.759098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59300 (local address 127.9.40.126:33203)
0809 02:15:04.759171 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.759174 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.759342 (+ 168us) server_negotiation.cc:407] Connection header received
0809 02:15:04.759427 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.759428 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.759438 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.759489 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.760269 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.760839 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.762022 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.762208 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.762265 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.765669 (+ 3404us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:59300: BlockingRecv error: recv got EOF from 127.0.0.1:59300 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Aug 09 02:15:04 dist-test-slave-qg60 krb5kdc[17694](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM
I20260809 02:15:04.775010 17819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.766740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59306 (local address 127.9.40.126:33203)
0809 02:15:04.766776 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.766778 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.766819 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:15:04.766906 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.766907 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.766915 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.766965 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:04.767771 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.768469 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.769540 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.769729 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.769786 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.772379 (+ 2593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:04.772381 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:04.772381 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:04.772390 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:04.773844 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.774219 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.774220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.774221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.774282 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:04.774484 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:04.774485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:04.774485 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:04.774607 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:04.774711 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.774815 (+ 104us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.774894 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:15:04.776605 17743 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:04.776810 17743 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:04.778257 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (DEFAULT_TABLE table=test-table [id=6021e62de0b4421fbc4410837d7a73ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.778326 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (DEFAULT_TABLE table=test-table [id=6021e62de0b4421fbc4410837d7a73ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.778326 18195 tablet_service.cc:1467] Processing CreateTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (DEFAULT_TABLE table=test-table [id=6021e62de0b4421fbc4410837d7a73ff]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.778412 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.778460 18195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.778460 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.779759 18298 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Bootstrap starting.
I20260809 02:15:04.779759 18300 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Bootstrap starting.
I20260809 02:15:04.779883 18299 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Bootstrap starting.
I20260809 02:15:04.780124 18300 tablet_bootstrap.cc:654] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.780138 18298 tablet_bootstrap.cc:654] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.780530 18299 tablet_bootstrap.cc:654] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.780961 18300 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: No bootstrap required, opened a new log
I20260809 02:15:04.781025 18300 ts_tablet_manager.cc:1397] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:04.781103 18298 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: No bootstrap required, opened a new log
I20260809 02:15:04.781142 18298 ts_tablet_manager.cc:1397] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:04.781147 18300 raft_consensus.cc:348] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.781222 18300 raft_consensus.cc:374] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.781244 18300 raft_consensus.cc:729] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.781286 18300 consensus_queue.cc:260] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.781287 18298 raft_consensus.cc:348] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.781348 18298 raft_consensus.cc:374] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.781368 18298 raft_consensus.cc:729] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.781369 18300 ts_tablet_manager.cc:1428] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:15:04.781420 18298 consensus_queue.cc:260] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.781522 18298 ts_tablet_manager.cc:1428] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.785181 18299 tablet_bootstrap.cc:492] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: No bootstrap required, opened a new log
I20260809 02:15:04.785251 18299 ts_tablet_manager.cc:1397] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.001s
I20260809 02:15:04.785458 18299 raft_consensus.cc:348] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.785565 18299 raft_consensus.cc:374] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.785591 18299 raft_consensus.cc:729] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.785646 18299 consensus_queue.cc:260] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.785758 18299 ts_tablet_manager.cc:1428] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:15:04.950771 18304 raft_consensus.cc:482] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:04.950886 18304 raft_consensus.cc:504] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.951117 18304 leader_election.cc:288] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.951398 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0d0cad2fe04dcbb4e6de3c5f2b4a57" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68" is_pre_election: true
I20260809 02:15:04.951404 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0d0cad2fe04dcbb4e6de3c5f2b4a57" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6" is_pre_election: true
I20260809 02:15:04.951516 18218 raft_consensus.cc:2454] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 0.
I20260809 02:15:04.951521 18067 raft_consensus.cc:2454] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 0.
I20260809 02:15:04.951705 17840 leader_election.cc:302] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9ecd4b34a2254b4bae566d98461328c6, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.951799 18304 raft_consensus.cc:2781] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:04.951841 18304 raft_consensus.cc:482] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:04.951871 18304 raft_consensus.cc:3037] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.952615 18304 raft_consensus.cc:504] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.952776 18304 leader_election.cc:288] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:04.952925 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0d0cad2fe04dcbb4e6de3c5f2b4a57" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68"
I20260809 02:15:04.952941 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a0d0cad2fe04dcbb4e6de3c5f2b4a57" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6"
I20260809 02:15:04.953022 18067 raft_consensus.cc:3037] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.953039 18218 raft_consensus.cc:3037] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:04.953665 18067 raft_consensus.cc:2454] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 1.
I20260809 02:15:04.953687 18218 raft_consensus.cc:2454] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 1.
I20260809 02:15:04.953846 17841 leader_election.cc:302] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:04.953963 18304 raft_consensus.cc:2781] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:04.954090 18304 raft_consensus.cc:686] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 1 LEADER]: Becoming Leader. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Running, Role: LEADER
I20260809 02:15:04.954188 18304 consensus_queue.cc:237] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.954967 17744 catalog_manager.cc:5495] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd5136614a8d4f9082c100c0525324e9 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "bd5136614a8d4f9082c100c0525324e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:04.966563 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:04.964092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.9.40.65:39083)
0809 02:15:04.964156 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:04.964158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:04.964192 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:15:04.964293 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:04.964294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:04.964313 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:04.964376 (+ 63us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:04.964484 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.965076 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.965846 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:04.966048 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:04.966106 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:04.966283 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:04.966388 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:04.966443 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:04.966487 (+ 44us) server_negotiation.cc:299] Negotiation successful
0809 02:15:04.966504 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:15:04.967638 18218 raft_consensus.cc:1261] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Refusing update from remote peer bd5136614a8d4f9082c100c0525324e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.967639 18067 raft_consensus.cc:1261] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Refusing update from remote peer bd5136614a8d4f9082c100c0525324e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:04.967926 18304 consensus_queue.cc:1035] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.968099 18379 consensus_queue.cc:1035] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:04.973237 17744 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.973332 17744 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:04.974071 17744 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Sending DeleteTablet for 3 replicas of tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57
I20260809 02:15:04.974359 18195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48944
I20260809 02:15:04.974370 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51088
I20260809 02:15:04.974493 18326 tablet_replica.cc:330] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: stopping tablet replica
I20260809 02:15:04.974493 18327 tablet_replica.cc:330] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: stopping tablet replica
I20260809 02:15:04.974553 18326 raft_consensus.cc:2229] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.974558 18327 raft_consensus.cc:2229] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:04.974612 18327 raft_consensus.cc:2258] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.974617 18326 raft_consensus.cc:2258] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:04.974936 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:04 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53404
I20260809 02:15:04.975039 18325 tablet_replica.cc:330] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: stopping tablet replica
I20260809 02:15:04.975080 18326 ts_tablet_manager.cc:1905] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.975095 18325 raft_consensus.cc:2229] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:04.975030 17744 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260809 02:15:04.975175 18325 raft_consensus.cc:2258] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:15:04.975183 17744 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:04.975772 18325 ts_tablet_manager.cc:1905] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.976392 18195 tablet_service.cc:1467] Processing CreateTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 (DEFAULT_TABLE table=test-table [id=5cf1c4b85fa34d58b09d2ba732aa25c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.976509 18195 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daf1657fed14a0ba50cb0eb2cb66259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.976835 17895 tablet_service.cc:1467] Processing CreateTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 (DEFAULT_TABLE table=test-table [id=5cf1c4b85fa34d58b09d2ba732aa25c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.976930 17895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daf1657fed14a0ba50cb0eb2cb66259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.977342 18327 ts_tablet_manager.cc:1905] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:04.977413 18325 ts_tablet_manager.cc:1918] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.977476 18325 log.cc:1196] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/7a0d0cad2fe04dcbb4e6de3c5f2b4a57
I20260809 02:15:04.977699 18325 ts_tablet_manager.cc:1939] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P bd5136614a8d4f9082c100c0525324e9: Deleting consensus metadata
I20260809 02:15:04.978106 18047 tablet_service.cc:1467] Processing CreateTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 (DEFAULT_TABLE table=test-table [id=5cf1c4b85fa34d58b09d2ba732aa25c8]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:04.978168 18299 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: Bootstrap starting.
I20260809 02:15:04.978212 18047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7daf1657fed14a0ba50cb0eb2cb66259. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:04.978583 18299 tablet_bootstrap.cc:654] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.978950 18326 ts_tablet_manager.cc:1918] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.979022 18326 log.cc:1196] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/7a0d0cad2fe04dcbb4e6de3c5f2b4a57
I20260809 02:15:04.979027 18327 ts_tablet_manager.cc:1918] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:04.979084 18327 log.cc:1196] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/7a0d0cad2fe04dcbb4e6de3c5f2b4a57
I20260809 02:15:04.979247 18326 ts_tablet_manager.cc:1939] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 9ecd4b34a2254b4bae566d98461328c6: Deleting consensus metadata
I20260809 02:15:04.979303 18327 ts_tablet_manager.cc:1939] T 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting consensus metadata
I20260809 02:15:04.979842 18299 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: No bootstrap required, opened a new log
I20260809 02:15:04.979890 18299 ts_tablet_manager.cc:1397] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260809 02:15:04.980067 18299 raft_consensus.cc:348] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.980139 18299 raft_consensus.cc:374] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.980155 18299 raft_consensus.cc:729] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.980183 18299 consensus_queue.cc:260] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.980273 18299 ts_tablet_manager.cc:1428] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:15:04.980602 18298 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Bootstrap starting.
I20260809 02:15:04.980809 18300 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Bootstrap starting.
I20260809 02:15:04.980794 17728 catalog_manager.cc:4841] TS bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083): tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (table test-table [id=6021e62de0b4421fbc4410837d7a73ff]) successfully deleted
I20260809 02:15:04.981158 18300 tablet_bootstrap.cc:654] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.981472 17728 catalog_manager.cc:4841] TS 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119): tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (table test-table [id=6021e62de0b4421fbc4410837d7a73ff]) successfully deleted
I20260809 02:15:04.981683 18298 tablet_bootstrap.cc:654] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:04.982113 18300 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: No bootstrap required, opened a new log
I20260809 02:15:04.982165 18300 ts_tablet_manager.cc:1397] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:04.982275 18300 raft_consensus.cc:348] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.982345 18300 raft_consensus.cc:374] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.982366 18300 raft_consensus.cc:729] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9ecd4b34a2254b4bae566d98461328c6, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.982409 18300 consensus_queue.cc:260] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.982497 18300 ts_tablet_manager.cc:1428] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.982833 18298 tablet_bootstrap.cc:492] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: No bootstrap required, opened a new log
I20260809 02:15:04.982887 18298 ts_tablet_manager.cc:1397] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:04.983014 18298 raft_consensus.cc:348] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.983091 18298 raft_consensus.cc:374] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:04.983114 18298 raft_consensus.cc:729] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ab8f35d91c5449a83e4277788cb4b68, State: Initialized, Role: FOLLOWER
I20260809 02:15:04.983157 18298 consensus_queue.cc:260] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:04.983248 18298 ts_tablet_manager.cc:1428] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:04.983603 17729 catalog_manager.cc:4841] TS 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973): tablet 7a0d0cad2fe04dcbb4e6de3c5f2b4a57 (table test-table [id=6021e62de0b4421fbc4410837d7a73ff]) successfully deleted
I20260809 02:15:05.159850 18388 raft_consensus.cc:482] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:05.160054 18388 raft_consensus.cc:504] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:05.160270 18388 leader_election.cc:288] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:05.160504 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daf1657fed14a0ba50cb0eb2cb66259" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68" is_pre_election: true
I20260809 02:15:05.160534 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daf1657fed14a0ba50cb0eb2cb66259" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6" is_pre_election: true
I20260809 02:15:05.160630 18218 raft_consensus.cc:2454] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 0.
I20260809 02:15:05.160631 18067 raft_consensus.cc:2454] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 0.
I20260809 02:15:05.160813 17841 leader_election.cc:302] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:05.160934 18388 raft_consensus.cc:2781] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:05.160977 18388 raft_consensus.cc:482] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:05.160998 18388 raft_consensus.cc:3037] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:05.161798 18388 raft_consensus.cc:504] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:05.161938 18388 leader_election.cc:288] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Requested vote from peers 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973), 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119)
I20260809 02:15:05.162150 18067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daf1657fed14a0ba50cb0eb2cb66259" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ab8f35d91c5449a83e4277788cb4b68"
I20260809 02:15:05.162237 18067 raft_consensus.cc:3037] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:05.162200 18218 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7daf1657fed14a0ba50cb0eb2cb66259" candidate_uuid: "bd5136614a8d4f9082c100c0525324e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9ecd4b34a2254b4bae566d98461328c6"
I20260809 02:15:05.162307 18218 raft_consensus.cc:3037] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:05.162874 18067 raft_consensus.cc:2454] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 1.
I20260809 02:15:05.163094 18218 raft_consensus.cc:2454] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bd5136614a8d4f9082c100c0525324e9 in term 1.
I20260809 02:15:05.163105 17841 leader_election.cc:302] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5ab8f35d91c5449a83e4277788cb4b68, bd5136614a8d4f9082c100c0525324e9; no voters:
I20260809 02:15:05.163218 18388 raft_consensus.cc:2781] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:05.163268 18388 raft_consensus.cc:686] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 1 LEADER]: Becoming Leader. State: Replica: bd5136614a8d4f9082c100c0525324e9, State: Running, Role: LEADER
I20260809 02:15:05.163316 18388 consensus_queue.cc:237] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } }
I20260809 02:15:05.163930 17744 catalog_manager.cc:5495] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to bd5136614a8d4f9082c100c0525324e9 (127.9.40.65). New cstate: current_term: 1 leader_uuid: "bd5136614a8d4f9082c100c0525324e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bd5136614a8d4f9082c100c0525324e9" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39083 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:05.173121 18310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:05.170830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56979 (local address 127.9.40.66:38973)
0809 02:15:05.170892 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:05.170894 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:05.170957 (+ 63us) server_negotiation.cc:407] Connection header received
0809 02:15:05.171004 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:05.171005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:05.171016 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:05.171067 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:05.171224 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.171688 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.172429 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.172619 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.172672 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:05.172896 (+ 224us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:05.172923 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:05.172962 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:05.173048 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 02:15:05.173064 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:15:05.173143 18396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:05.170701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:38973 (local address 127.9.40.65:56979)
0809 02:15:05.170870 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:05.170883 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:05.170960 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:05.171108 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:05.171109 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:05.171111 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:05.171159 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:05.171160 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.171761 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.171762 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:05.172355 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:05.172357 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.172694 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.172695 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:05.172799 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:05.172822 (+ 23us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:05.173002 (+ 180us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:05.173004 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:15:05.173038 (+ 34us) client_negotiation.cc:240] Negotiation successful
0809 02:15:05.173082 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":56,"threads_started":1}
I20260809 02:15:05.175356 18218 raft_consensus.cc:1261] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Refusing update from remote peer bd5136614a8d4f9082c100c0525324e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:05.175395 18067 raft_consensus.cc:1261] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Refusing update from remote peer bd5136614a8d4f9082c100c0525324e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:05.175578 18388 consensus_queue.cc:1035] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9ecd4b34a2254b4bae566d98461328c6" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 46119 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:05.175679 18379 consensus_queue.cc:1035] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5ab8f35d91c5449a83e4277788cb4b68" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 38973 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:05.183004 18362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:05.180110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39083 (local address 127.9.40.66:46817)
0809 02:15:05.180154 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:05.180167 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:05.180211 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:05.180351 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:05.180351 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:05.180352 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0809 02:15:05.180393 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:05.180394 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.181344 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.181345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:05.181906 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:05.181957 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.182542 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.182543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:05.182614 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:05.182621 (+ 7us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0809 02:15:05.182833 (+ 212us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0809 02:15:05.182835 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:15:05.182883 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:15:05.182935 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 02:15:05.182991 18292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:05.180209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46817 (local address 127.9.40.65:39083)
0809 02:15:05.180254 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:05.180256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:05.180264 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:05.180282 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:05.180283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:05.180292 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:05.180346 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:05.180460 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.181149 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.182101 (+ 952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:05.182446 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:05.182547 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:05.182723 (+ 176us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:05.182760 (+ 37us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:05.182831 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:05.182885 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 02:15:05.182910 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:15:05.189123 17743 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:05.189224 17743 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59306:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:05.189927 17743 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ac0f7671b22047cca1af257cad142f7e: Sending DeleteTablet for 3 replicas of tablet 7daf1657fed14a0ba50cb0eb2cb66259
I20260809 02:15:05.190253 17895 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:05 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:53404
I20260809 02:15:05.190301 18195 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:05 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48944
I20260809 02:15:05.190392 18325 tablet_replica.cc:330] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9: stopping tablet replica
I20260809 02:15:05.190456 18325 raft_consensus.cc:2229] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:05.190521 18325 raft_consensus.cc:2258] T 7daf1657fed14a0ba50cb0eb2cb66259 P bd5136614a8d4f9082c100c0525324e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:05.190719 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17824
I20260809 02:15:05.191843 18326 tablet_replica.cc:330] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: stopping tablet replica
I20260809 02:15:05.191926 18326 raft_consensus.cc:2229] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:05.191974 18326 raft_consensus.cc:2258] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:05.192981 18047 tablet_service.cc:1514] Processing DeleteTablet for tablet 7daf1657fed14a0ba50cb0eb2cb66259 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:05 UTC) from {username='oryx', principal='oryx/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51088
I20260809 02:15:05.193347 18326 ts_tablet_manager.cc:1905] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:05.195137 18326 ts_tablet_manager.cc:1918] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:05.195199 18326 log.cc:1196] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/7daf1657fed14a0ba50cb0eb2cb66259
I20260809 02:15:05.195442 18326 ts_tablet_manager.cc:1939] T 7daf1657fed14a0ba50cb0eb2cb66259 P 9ecd4b34a2254b4bae566d98461328c6: Deleting consensus metadata
I20260809 02:15:05.195792 17728 catalog_manager.cc:4841] TS 9ecd4b34a2254b4bae566d98461328c6 (127.9.40.67:46119): tablet 7daf1657fed14a0ba50cb0eb2cb66259 (table test-table [id=5cf1c4b85fa34d58b09d2ba732aa25c8]) successfully deleted
I20260809 02:15:05.195881 18327 tablet_replica.cc:330] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: stopping tablet replica
I20260809 02:15:05.195966 18327 raft_consensus.cc:2229] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:05.196010 18327 raft_consensus.cc:2258] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:05.196560 18327 ts_tablet_manager.cc:1905] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:05.198228 18327 ts_tablet_manager.cc:1918] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260809 02:15:05.198289 18327 log.cc:1196] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/7daf1657fed14a0ba50cb0eb2cb66259
I20260809 02:15:05.198534 18327 ts_tablet_manager.cc:1939] T 7daf1657fed14a0ba50cb0eb2cb66259 P 5ab8f35d91c5449a83e4277788cb4b68: Deleting consensus metadata
I20260809 02:15:05.199093 17729 catalog_manager.cc:4841] TS 5ab8f35d91c5449a83e4277788cb4b68 (127.9.40.66:38973): tablet 7daf1657fed14a0ba50cb0eb2cb66259 (table test-table [id=5cf1c4b85fa34d58b09d2ba732aa25c8]) successfully deleted
W20260809 02:15:05.205927 17728 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:39083 (error 108)
I20260809 02:15:05.206192 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17975
I20260809 02:15:05.207937 18402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:05.206303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39083 (local address 127.0.0.1:53414)
0809 02:15:05.207818 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:05.207884 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39083: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1493,"thread_start_us":37,"threads_started":1}
W20260809 02:15:05.208087 17728 catalog_manager.cc:4584] TS bd5136614a8d4f9082c100c0525324e9 (127.9.40.65:39083): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7daf1657fed14a0ba50cb0eb2cb66259: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39083: connect: Connection refused (error 111)
I20260809 02:15:05.214893 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18128
I20260809 02:15:05.222443 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 17709
2026-08-09T02:15:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2986 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:05 dist-test-slave-qg60 krb5kdc[18405](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:05 dist-test-slave-qg60 krb5kdc[18405](info): set up 2 sockets
Aug 09 02:15:05 dist-test-slave-qg60 krb5kdc[18405](info): commencing operation
krb5kdc: starting...
W20260809 02:15:06.358059 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.112s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:15:06 dist-test-slave-qg60 krb5kdc[18405](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:15:06.486039 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35069
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:45939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35069
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:06.569272 18419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:06.569533 18419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:06.569573 18419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:06.570713 18419 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:06.570785 18419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:06.570811 18419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:06.570832 18419 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:06.570863 18419 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:06.572414 18419 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:45939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35069
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35069
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:06.572710 18419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:06.572978 18419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:06.578202 18426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:06.578195 18425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:06.578210 18428 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:06.578305 18419 server_base.cc:1034] running on GCE node
I20260809 02:15:06.578655 18419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:06.578951 18419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:06.580118 18419 hybrid_clock.cc:648] HybridClock initialized: now 1786241706580102 us; error 28 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-09T02:15:06Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1358 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:06 dist-test-slave-qg60 krb5kdc[18433](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:06 dist-test-slave-qg60 krb5kdc[18433](info): set up 2 sockets
Aug 09 02:15:06 dist-test-slave-qg60 krb5kdc[18433](info): commencing operation
krb5kdc: starting...
W20260809 02:15:07.729602 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.124s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:15:07 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:15:07.861094 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33273
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:07.945676 18446 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:07.945922 18446 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:07.945961 18446 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:07.947181 18446 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:07.947252 18446 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:07.947275 18446 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:07.947296 18446 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:07.947324 18446 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:07.948844 18446 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18446
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:07.949155 18446 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:07.949393 18446 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:07.954392 18452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:07.954375 18454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:07.954476 18446 server_base.cc:1034] running on GCE node
W20260809 02:15:07.954376 18451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:07.954847 18446 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:07.955148 18446 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:07.956306 18446 hybrid_clock.cc:648] HybridClock initialized: now 1786241707956294 us; error 28 us; skew 500 ppm
Aug 09 02:15:07 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241707, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:07.959082 18446 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:15:07.959607 18446 webserver.cc:460] Webserver started at http://127.9.40.126:33895/ using document root <none> and password file <none>
I20260809 02:15:07.959867 18446 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:07.959937 18446 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:07.960109 18446 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:07.961258 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "50c6b8400b6549698127d3e7c4d076b0"
format_stamp: "Formatted at 2026-08-09 02:15:07 on dist-test-slave-qg60"
server_key: "a2230542959fe5bc29027cb1fa6939ef"
server_key_iv: "9d0850634b4da6a5bcabb7902ef207ca"
server_key_version: "encryptionkey@0"
I20260809 02:15:07.961628 18446 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "50c6b8400b6549698127d3e7c4d076b0"
format_stamp: "Formatted at 2026-08-09 02:15:07 on dist-test-slave-qg60"
server_key: "a2230542959fe5bc29027cb1fa6939ef"
server_key_iv: "9d0850634b4da6a5bcabb7902ef207ca"
server_key_version: "encryptionkey@0"
I20260809 02:15:07.963065 18446 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:15:07.963922 18462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:07.964185 18446 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:07.964301 18446 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "50c6b8400b6549698127d3e7c4d076b0"
format_stamp: "Formatted at 2026-08-09 02:15:07 on dist-test-slave-qg60"
server_key: "a2230542959fe5bc29027cb1fa6939ef"
server_key_iv: "9d0850634b4da6a5bcabb7902ef207ca"
server_key_version: "encryptionkey@0"
I20260809 02:15:07.964380 18446 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:07.987362 18446 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:07.989993 18446 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:07.990187 18446 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:07.994851 18446 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33273
I20260809 02:15:07.994889 18517 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33273 every 8 connection(s)
I20260809 02:15:07.995276 18446 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:07.996228 18518 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:07.997051 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18446
I20260809 02:15:07.997169 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:15:07.997370 9377 external_mini_cluster.cc:1428] Setting key 88092f68bfb5cf960328569bd04313c5
I20260809 02:15:07.999696 18518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Bootstrap starting.
I20260809 02:15:08.000334 18518 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:08.000643 18518 log.cc:824] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:08.001652 18518 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: No bootstrap required, opened a new log
I20260809 02:15:08.004388 18518 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.004593 18518 raft_consensus.cc:374] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:08.004643 18518 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c6b8400b6549698127d3e7c4d076b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:08.004813 18518 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.004904 18518 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:08.004993 18518 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:08.005035 18518 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 0 FOLLOWER]: Advancing to term 1
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241707, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:08.005693 18518 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.005815 18518 leader_election.cc:302] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c6b8400b6549698127d3e7c4d076b0; no voters:
I20260809 02:15:08.006028 18518 leader_election.cc:288] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:08.006096 18523 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:08.006330 18518 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:08.006606 18523 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 LEADER]: Becoming Leader. State: Replica: 50c6b8400b6549698127d3e7c4d076b0, State: Running, Role: LEADER
I20260809 02:15:08.006722 18523 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.007596 18524 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "50c6b8400b6549698127d3e7c4d076b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } } }
I20260809 02:15:08.007695 18525 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c6b8400b6549698127d3e7c4d076b0. Latest consensus state: current_term: 1 leader_uuid: "50c6b8400b6549698127d3e7c4d076b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } } }
I20260809 02:15:08.007790 18525 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:08.007710 18524 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:08.008031 18539 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:08.008919 18539 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:08.010188 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:07.998007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41544 (local address 127.9.40.126:33273)
0809 02:15:07.998273 (+ 266us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:07.998290 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:07.998317 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:15:07.998736 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:07.998742 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:07.998788 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:07.998912 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:07.999770 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.001048 (+ 1278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.002292 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.002724 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.002831 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.005920 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:08.005923 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:08.005924 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:08.005955 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:08.008311 (+ 2356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.008871 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.008874 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.008874 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.009017 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.009255 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.009256 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.009257 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.009520 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:08.009668 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:08.009791 (+ 123us) server_negotiation.cc:299] Negotiation successful
0809 02:15:08.009939 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":103,"threads_started":1}
I20260809 02:15:08.010565 18539 catalog_manager.cc:1347] Generated new cluster ID: a51e17c1d8f24fbabee6c666b9dad387
I20260809 02:15:08.010624 18539 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:08.027456 18539 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:08.028043 18539 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:08.033739 18539 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Generated new TSK 0
I20260809 02:15:08.033957 18539 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:15:08.068162 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:08.155462 18545 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:08.155674 18545 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:08.155699 18545 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:08.156919 18545 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:08.156962 18545 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:08.156996 18545 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:08.158470 18545 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18545
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:08.158708 18545 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:08.158962 18545 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:08.164263 18554 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.164270 18551 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.164273 18552 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.164716 18545 server_base.cc:1034] running on GCE node
I20260809 02:15:08.164958 18545 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:08.165238 18545 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:08.166426 18545 hybrid_clock.cc:648] HybridClock initialized: now 1786241708166416 us; error 42 us; skew 500 ppm
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:08.169269 18545 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:15:08.169782 18545 webserver.cc:460] Webserver started at http://127.9.40.65:45375/ using document root <none> and password file <none>
I20260809 02:15:08.170043 18545 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:08.170116 18545 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:08.170269 18545 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:08.171372 18545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "41cb8868ac994ae194872505edf70e3a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "f04a521e6ba158ad51994a25b7b541be"
server_key_iv: "f93923223362672749e315c3f15040a7"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.171763 18545 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "41cb8868ac994ae194872505edf70e3a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "f04a521e6ba158ad51994a25b7b541be"
server_key_iv: "f93923223362672749e315c3f15040a7"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.173298 18545 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:08.174170 18562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.174399 18545 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:08.174531 18545 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "41cb8868ac994ae194872505edf70e3a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "f04a521e6ba158ad51994a25b7b541be"
server_key_iv: "f93923223362672749e315c3f15040a7"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.174615 18545 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:08.190469 18545 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:08.192719 18545 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:08.192924 18545 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:08.193200 18545 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:08.193524 18545 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:08.193579 18545 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.193615 18545 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:08.193658 18545 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.199505 18545 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:39713
I20260809 02:15:08.199592 18682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:39713 every 8 connection(s)
I20260809 02:15:08.199922 18545 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:08.204056 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18545
I20260809 02:15:08.204178 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:08.204382 9377 external_mini_cluster.cc:1428] Setting key da607834418b72877bb3600f9d9f6b94
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:08.210806 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.200719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40145 (local address 127.9.40.126:33273)
0809 02:15:08.200811 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.200813 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.201009 (+ 196us) server_negotiation.cc:407] Connection header received
0809 02:15:08.201598 (+ 589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.201600 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.201612 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.201660 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.202676 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.203217 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.204502 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.204768 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.204838 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.207395 (+ 2557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:08.207396 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:08.207396 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:08.207405 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:08.209230 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.209766 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.209767 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.209767 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.209816 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.210125 (+ 309us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.210125 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.210126 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.210229 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:08.210280 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:08.210660 (+ 380us) server_negotiation.cc:299] Negotiation successful
0809 02:15:08.210739 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:15:08.211131 18685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.200704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:40145)
0809 02:15:08.200880 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.200923 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.201475 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.201740 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.201747 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.202333 (+ 586us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.202531 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.202536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.203323 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.203324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.204381 (+ 1057us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.204384 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.204859 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.204860 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.205000 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.205626 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.205634 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.207295 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:08.209319 (+ 2024us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.209327 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.209329 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.209685 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.209881 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.209882 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.209883 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.210025 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.210303 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:08.210311 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:08.210509 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 02:15:08.210643 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 02:15:08.210837 (+ 194us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260809 02:15:08.211733 18683 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33273
I20260809 02:15:08.211833 18683 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:08.212038 18683 heartbeater.cc:511] Master 127.9.40.126:33273 requested a full tablet report, sending...
I20260809 02:15:08.212598 18482 ts_manager.cc:194] Registered new tserver with Master: 41cb8868ac994ae194872505edf70e3a (127.9.40.65:39713)
I20260809 02:15:08.213284 18482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:40145
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:15:08.233784 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:08.317499 18688 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:08.317744 18688 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:08.317783 18688 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:08.318943 18688 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:08.319020 18688 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:08.319075 18688 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:08.320604 18688 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18688
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:08.320925 18688 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:08.321187 18688 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:08.326630 18698 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.326625 18695 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.326740 18693 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.327028 18688 server_base.cc:1034] running on GCE node
I20260809 02:15:08.327245 18688 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:08.327484 18688 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:08.328626 18688 hybrid_clock.cc:648] HybridClock initialized: now 1786241708328613 us; error 28 us; skew 500 ppm
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:08.331339 18688 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:15:08.331815 18688 webserver.cc:460] Webserver started at http://127.9.40.66:35253/ using document root <none> and password file <none>
I20260809 02:15:08.332065 18688 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:08.332144 18688 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:08.332293 18688 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:08.333447 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "7710e56087a74206be526e1ecf24f545"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "650632b0bcc27cb3ac23f46305e66879"
server_key_iv: "bfbea6c5f9086b479e020096f9a08636"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.333802 18688 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "7710e56087a74206be526e1ecf24f545"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "650632b0bcc27cb3ac23f46305e66879"
server_key_iv: "bfbea6c5f9086b479e020096f9a08636"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.335273 18688 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:15:08.335990 18705 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.336172 18688 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:15:08.336275 18688 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "7710e56087a74206be526e1ecf24f545"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "650632b0bcc27cb3ac23f46305e66879"
server_key_iv: "bfbea6c5f9086b479e020096f9a08636"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.336349 18688 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:08.345938 18688 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:08.347931 18688 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:08.348116 18688 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:08.348353 18688 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:08.348673 18688 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:08.348729 18688 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.348856 18688 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:08.348887 18688 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.354763 18688 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:36689
I20260809 02:15:08.354808 18826 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:36689 every 8 connection(s)
I20260809 02:15:08.355136 18688 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:08.359824 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18688
I20260809 02:15:08.359958 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:08.360204 9377 external_mini_cluster.cc:1428] Setting key 4f2c189a96e856998609de492fcc4253
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:08.365399 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.355851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:51999 (local address 127.9.40.126:33273)
0809 02:15:08.355925 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.355927 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.356166 (+ 239us) server_negotiation.cc:407] Connection header received
0809 02:15:08.356654 (+ 488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.356655 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.356666 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.356720 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.357776 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.358324 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.359324 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.359500 (+ 176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.359554 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.362418 (+ 2864us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:08.362419 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:08.362420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:08.362429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:08.363840 (+ 1411us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.364319 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.364320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.364321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.364388 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.364666 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.364666 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.364667 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.364849 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:08.364928 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:08.365213 (+ 285us) server_negotiation.cc:299] Negotiation successful
0809 02:15:08.365299 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260809 02:15:08.365612 18829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.355835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:51999)
0809 02:15:08.355988 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.356055 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.356564 (+ 509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.356843 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.356846 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.357461 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.357681 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.357686 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.358414 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.358415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.359246 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.359249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.359666 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.359667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.359797 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.360665 (+ 868us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.360676 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.362329 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:08.363969 (+ 1640us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.363976 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.363978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.364203 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.364470 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.364471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.364472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.364584 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.364925 (+ 341us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:08.364934 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:08.365075 (+ 141us) client_negotiation.cc:769] Sending connection context
0809 02:15:08.365189 (+ 114us) client_negotiation.cc:240] Negotiation successful
0809 02:15:08.365364 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":71,"threads_started":1}
I20260809 02:15:08.366187 18827 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33273
I20260809 02:15:08.366302 18827 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:08.366513 18827 heartbeater.cc:511] Master 127.9.40.126:33273 requested a full tablet report, sending...
I20260809 02:15:08.366966 18482 ts_manager.cc:194] Registered new tserver with Master: 7710e56087a74206be526e1ecf24f545 (127.9.40.66:36689)
I20260809 02:15:08.367503 18482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:51999
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:15:08.389171 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:08.474963 18832 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:08.475196 18832 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:08.475224 18832 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:08.476471 18832 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:08.476511 18832 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:08.476540 18832 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:08.478268 18832 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18832
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:08.478482 18832 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:08.478669 18832 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:08.484045 18837 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.484052 18838 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.484071 18840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.484279 18832 server_base.cc:1034] running on GCE node
I20260809 02:15:08.484503 18832 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:08.484813 18832 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:08.485996 18832 hybrid_clock.cc:648] HybridClock initialized: now 1786241708485981 us; error 52 us; skew 500 ppm
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:08.488569 18832 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:15:08.489040 18832 webserver.cc:460] Webserver started at http://127.9.40.67:44045/ using document root <none> and password file <none>
I20260809 02:15:08.489231 18832 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:08.489271 18832 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:08.489362 18832 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:08.490361 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "f3ce08be65a94ad89b8540dd8661c15a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "ac7d9df3785bde28395a3a6aa1733ec0"
server_key_iv: "fe4ba2072c573ac596369aa6d4b19d1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.490661 18832 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "f3ce08be65a94ad89b8540dd8661c15a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "ac7d9df3785bde28395a3a6aa1733ec0"
server_key_iv: "fe4ba2072c573ac596369aa6d4b19d1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.492151 18832 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:15:08.492981 18847 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.493206 18832 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:08.493306 18832 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "f3ce08be65a94ad89b8540dd8661c15a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "ac7d9df3785bde28395a3a6aa1733ec0"
server_key_iv: "fe4ba2072c573ac596369aa6d4b19d1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.493364 18832 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:08.520556 18832 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:08.522868 18832 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:08.523005 18832 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:08.523226 18832 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:08.523623 18832 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:08.523679 18832 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.523739 18832 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:08.523768 18832 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.530325 18832 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38459
I20260809 02:15:08.530402 18971 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38459 every 8 connection(s)
I20260809 02:15:08.530804 18832 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:08.535544 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18832
I20260809 02:15:08.535663 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:08.535852 9377 external_mini_cluster.cc:1428] Setting key 8657b7d95271f402137010408b5914ea
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:08.541414 18521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.531649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33363 (local address 127.9.40.126:33273)
0809 02:15:08.531701 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.531703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.531936 (+ 233us) server_negotiation.cc:407] Connection header received
0809 02:15:08.532450 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.532451 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.532461 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.532531 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.533626 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.534288 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.535397 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.535583 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.535770 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.538287 (+ 2517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:08.538288 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:08.538289 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:08.538296 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:08.539832 (+ 1536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.540262 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.540263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.540264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.540351 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:08.540595 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:08.540596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:08.540596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:08.540708 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:08.540835 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:08.541221 (+ 386us) server_negotiation.cc:299] Negotiation successful
0809 02:15:08.541320 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:15:08.541615 18974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.531613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:33363)
0809 02:15:08.531791 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.531838 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.532355 (+ 517us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.532592 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.532595 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.533299 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.533523 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.533529 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.534365 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.534366 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.535297 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.535300 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.535781 (+ 481us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.535782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.535917 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.536652 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.536662 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.538173 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:08.539915 (+ 1742us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.539922 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.539923 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.540165 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.540418 (+ 253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:08.540419 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:08.540419 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:08.540528 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:08.540830 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:08.540837 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:08.540985 (+ 148us) client_negotiation.cc:769] Sending connection context
0809 02:15:08.541176 (+ 191us) client_negotiation.cc:240] Negotiation successful
0809 02:15:08.541362 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":100,"threads_started":1}
I20260809 02:15:08.542151 18972 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33273
I20260809 02:15:08.542249 18972 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:08.542393 18972 heartbeater.cc:511] Master 127.9.40.126:33273 requested a full tablet report, sending...
I20260809 02:15:08.542749 18482 ts_manager.cc:194] Registered new tserver with Master: f3ce08be65a94ad89b8540dd8661c15a (127.9.40.67:38459)
I20260809 02:15:08.543095 18482 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:33363
I20260809 02:15:08.547978 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
Entry for principal oryx/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab.
WARNING: no policy specified for oryx/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
Entry for principal oryx/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab.
WARNING: no policy specified for oryx/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
Entry for principal oryx/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab.
I20260809 02:15:08.631608 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18545
I20260809 02:15:08.638696 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18688
I20260809 02:15:08.649755 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18832
I20260809 02:15:08.656067 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18446
I20260809 02:15:08.665624 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33273
--webserver_interface=127.9.40.126
--webserver_port=33895
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:08.754001 18981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:08.754323 18981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:08.754366 18981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:08.755640 18981 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:08.755716 18981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:08.755743 18981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:08.755772 18981 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:08.755798 18981 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:08.757478 18981 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=33895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:08.757776 18981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:08.758020 18981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:08.763260 18989 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.763319 18987 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.763291 18986 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.763507 18981 server_base.cc:1034] running on GCE node
I20260809 02:15:08.763772 18981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:08.764099 18981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:08.765300 18981 hybrid_clock.cc:648] HybridClock initialized: now 1786241708765285 us; error 48 us; skew 500 ppm
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:08.771584 18981 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:15:08.772120 18981 webserver.cc:460] Webserver started at http://127.9.40.126:33895/ using document root <none> and password file <none>
I20260809 02:15:08.772377 18981 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:08.772488 18981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:08.774016 18981 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:08.774845 18996 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.775079 18981 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:08.775193 18981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "50c6b8400b6549698127d3e7c4d076b0"
format_stamp: "Formatted at 2026-08-09 02:15:07 on dist-test-slave-qg60"
server_key: "a2230542959fe5bc29027cb1fa6939ef"
server_key_iv: "9d0850634b4da6a5bcabb7902ef207ca"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.775609 18981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:08.790802 18981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:08.793260 18981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:08.793454 18981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:08.798310 18981 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33273
I20260809 02:15:08.798398 19051 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33273 every 8 connection(s)
I20260809 02:15:08.798841 18981 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:08.800379 19052 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:15:08.801013 19052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Bootstrap starting.
I20260809 02:15:08.801658 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 18981
I20260809 02:15:08.802002 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:39713
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:33273
--webserver_port=45375
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:15:08.810562 19052 log.cc:824] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:08.812279 19052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:15:08.812597 19052 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Bootstrap complete.
I20260809 02:15:08.814972 19052 raft_consensus.cc:348] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.815194 19052 raft_consensus.cc:729] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 50c6b8400b6549698127d3e7c4d076b0, State: Initialized, Role: FOLLOWER
I20260809 02:15:08.815356 19052 consensus_queue.cc:260] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.815430 19052 raft_consensus.cc:388] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:08.815498 19052 raft_consensus.cc:482] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:08.815543 19052 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:15:08.816603 19052 raft_consensus.cc:504] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.816690 19052 leader_election.cc:302] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 50c6b8400b6549698127d3e7c4d076b0; no voters:
I20260809 02:15:08.816947 19052 leader_election.cc:288] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:15:08.817034 19056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:15:08.817203 19056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [term 2 LEADER]: Becoming Leader. State: Replica: 50c6b8400b6549698127d3e7c4d076b0, State: Running, Role: LEADER
I20260809 02:15:08.817270 19052 sys_catalog.cc:564] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:08.817353 19056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } }
I20260809 02:15:08.817624 19056 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 50c6b8400b6549698127d3e7c4d076b0. Latest consensus state: current_term: 2 leader_uuid: "50c6b8400b6549698127d3e7c4d076b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } } }
I20260809 02:15:08.817705 19056 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:08.817653 19057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "50c6b8400b6549698127d3e7c4d076b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "50c6b8400b6549698127d3e7c4d076b0" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33273 } } }
I20260809 02:15:08.817898 19057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:08.817922 19063 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:08.818478 19063 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:08.818638 19063 catalog_manager.cc:1259] Loaded cluster ID: a51e17c1d8f24fbabee6c666b9dad387
I20260809 02:15:08.818681 19063 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:08.819859 19063 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:08.820058 19063 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 50c6b8400b6549698127d3e7c4d076b0: Loaded TSK: 0
I20260809 02:15:08.820271 19063 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:15:08.892596 19054 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:08.892904 19054 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:08.892951 19054 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:08.894145 19054 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:08.894223 19054 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:08.894271 19054 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:08.895784 19054 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.65.keytab
--principal=oryx/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:39713
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=45375
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19054
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:08.896078 19054 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:08.896333 19054 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:08.901906 19079 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.902099 19054 server_base.cc:1034] running on GCE node
W20260809 02:15:08.901897 19082 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:08.901897 19080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:08.902449 19054 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:08.902730 19054 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:08.903891 19054 hybrid_clock.cc:648] HybridClock initialized: now 1786241708903870 us; error 37 us; skew 500 ppm
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241708, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:08.906626 19054 init.cc:377] Logged in from keytab as oryx/127.9.40.65@KRBTEST.COM (short username oryx)
I20260809 02:15:08.907163 19054 webserver.cc:460] Webserver started at http://127.9.40.65:45375/ using document root <none> and password file <none>
I20260809 02:15:08.907433 19054 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:08.907497 19054 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:08.909053 19054 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:08.909704 19091 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.909900 19054 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:15:08.910002 19054 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "41cb8868ac994ae194872505edf70e3a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "f04a521e6ba158ad51994a25b7b541be"
server_key_iv: "f93923223362672749e315c3f15040a7"
server_key_version: "encryptionkey@0"
I20260809 02:15:08.910430 19054 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:08.920050 19054 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:08.922493 19054 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:08.922672 19054 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:08.922937 19054 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:08.923311 19054 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:08.923369 19054 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.923413 19054 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:08.923440 19054 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:08.931195 19054 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:39713
I20260809 02:15:08.931282 19211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:39713 every 8 connection(s)
I20260809 02:15:08.931655 19054 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:08.938583 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19054
I20260809 02:15:08.939204 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:36689
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:33273
--webserver_port=35253
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:08.942003 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.932464 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:50675)
0809 02:15:08.932692 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.932758 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.933431 (+ 673us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.933906 (+ 475us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.933910 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.934682 (+ 772us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.934948 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.934954 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.936241 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.936242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.937604 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.937607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.938264 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.938265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.938440 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.939056 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.939067 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.941765 (+ 2698us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":102,"threads_started":1}
W20260809 02:15:08.942040 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:08.932603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:50675 (local address 127.9.40.126:33273)
0809 02:15:08.932800 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.932809 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.932849 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:15:08.933672 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.933677 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.933719 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.933829 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.935097 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.936141 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.937723 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.938159 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.938260 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.941891 (+ 3631us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:50675: BlockingRecv error: recv got EOF from 127.9.40.65:50675 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
W20260809 02:15:08.942287 19212 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:33273 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:33273: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:08.947089 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.942523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:45465)
0809 02:15:08.942586 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.942599 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.942668 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.942819 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.942820 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.942968 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.943020 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.943022 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.943658 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.943659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.944157 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.944158 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.944607 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.944608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.944679 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.945244 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.945249 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.947001 (+ 1752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":41}
W20260809 02:15:08.947144 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:08.942617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45465 (local address 127.9.40.126:33273)
0809 02:15:08.942671 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.942673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.942682 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:08.942730 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.942731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.942743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.942818 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.943100 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.943576 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.944241 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.944478 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.944594 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.947078 (+ 2484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:45465: BlockingRecv error: recv got EOF from 127.9.40.65:45465 (error 108)
Metrics: {"server-negotiator.queue_time_us":33}
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:08 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:08.952612 19215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:08.947676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:40219)
0809 02:15:08.947760 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:08.947846 (+ 86us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:08.947912 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:08.948125 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:08.948126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:08.948277 (+ 151us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:08.948308 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.948310 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.949101 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.949102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.949659 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:08.949661 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.950094 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.950095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:08.950164 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.950653 (+ 489us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:08.950659 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:08.952513 (+ 1854us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:15:08.952670 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:08.947809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40219 (local address 127.9.40.126:33273)
0809 02:15:08.947850 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:08.947853 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:08.947895 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:15:08.948027 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:08.948028 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:08.948042 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:08.948120 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:08.948403 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.949009 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.949771 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:08.949981 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:08.950094 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:08.952595 (+ 2501us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:40219: BlockingRecv error: recv got EOF from 127.9.40.65:40219 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:15:08.952961 19212 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:15:09.029316 19217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:09.029537 19217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:09.029563 19217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:09.030788 19217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:09.030836 19217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:09.030865 19217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:09.032384 19217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.66.keytab
--principal=oryx/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:36689
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=35253
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:09.032635 19217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:09.032929 19217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:09.038244 19222 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:09.038295 19226 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:09.038300 19224 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:09.038659 19217 server_base.cc:1034] running on GCE node
I20260809 02:15:09.038868 19217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:09.039126 19217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:09.040319 19217 hybrid_clock.cc:648] HybridClock initialized: now 1786241709040303 us; error 28 us; skew 500 ppm
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241709, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:09.043278 19217 init.cc:377] Logged in from keytab as oryx/127.9.40.66@KRBTEST.COM (short username oryx)
I20260809 02:15:09.043794 19217 webserver.cc:460] Webserver started at http://127.9.40.66:35253/ using document root <none> and password file <none>
I20260809 02:15:09.044121 19217 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:09.044221 19217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:09.045759 19217 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:09.046470 19233 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.046644 19217 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:15:09.046738 19217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "7710e56087a74206be526e1ecf24f545"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "650632b0bcc27cb3ac23f46305e66879"
server_key_iv: "bfbea6c5f9086b479e020096f9a08636"
server_key_version: "encryptionkey@0"
I20260809 02:15:09.047169 19217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:09.057173 19217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:09.059296 19217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:09.059479 19217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:09.059742 19217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:09.060065 19217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:09.060120 19217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.060164 19217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:09.060197 19217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.066426 19217 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:36689
I20260809 02:15:09.066474 19357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:36689 every 8 connection(s)
I20260809 02:15:09.066812 19217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:09.067399 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19217
I20260809 02:15:09.067761 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:38459
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:33273
--webserver_port=44045
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:36311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.075793 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.067578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:46625)
0809 02:15:09.067771 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.067824 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.068662 (+ 838us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.068972 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.068976 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.069828 (+ 852us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.070022 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.070027 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.071084 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.071085 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.071899 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.071901 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.072521 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.072522 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.072629 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.073277 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.073285 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.075646 (+ 2361us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":94,"threads_started":1}
W20260809 02:15:09.075856 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.067576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46625 (local address 127.9.40.126:33273)
0809 02:15:09.067661 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.067663 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.067918 (+ 255us) server_negotiation.cc:407] Connection header received
0809 02:15:09.068801 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.068803 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.068821 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.068885 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.070125 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.070941 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.072072 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.072416 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.072495 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.075771 (+ 3276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:46625: BlockingRecv error: recv got EOF from 127.9.40.66:46625 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260809 02:15:09.076103 19358 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:33273 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:33273: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.081135 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.076384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:44859)
0809 02:15:09.076434 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.076447 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.076521 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.076681 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.076682 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.076851 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.076893 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.076895 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.077506 (+ 611us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.077507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.078153 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.078155 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.078518 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.078519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.078602 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.079139 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.079144 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.081061 (+ 1917us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 02:15:09.081291 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.076432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44859 (local address 127.9.40.126:33273)
0809 02:15:09.076496 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.076498 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.076508 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:09.076583 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.076584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.076593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.076673 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.076973 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.077424 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.078250 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.078436 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.078508 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.081211 (+ 2703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:44859: BlockingRecv error: recv got EOF from 127.9.40.66:44859 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.086048 19361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.081535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:44243)
0809 02:15:09.081728 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.081736 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.081794 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.082029 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.082030 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.082214 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.082245 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.082247 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.082880 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.082881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.083360 (+ 479us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.083361 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.083720 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.083721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.083774 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.084201 (+ 427us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.084205 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.085962 (+ 1757us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181}
W20260809 02:15:09.086107 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.081591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44243 (local address 127.9.40.126:33273)
0809 02:15:09.081644 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.081646 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.081781 (+ 135us) server_negotiation.cc:407] Connection header received
0809 02:15:09.081853 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.081854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.081863 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.082020 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.082337 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.082800 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.083466 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.083647 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.083722 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.086039 (+ 2317us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:44243: BlockingRecv error: recv got EOF from 127.9.40.66:44243 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
W20260809 02:15:09.086447 19358 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:15:09.153852 19362 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:09.154057 19362 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:09.154081 19362 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:09.155170 19362 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:09.155206 19362 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:09.155231 19362 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:09.156590 19362 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:36311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/krb5kdc/oryx_127.9.40.67.keytab
--principal=oryx/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:38459
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=44045
--tserver_master_addrs=127.9.40.126:33273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19362
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:09.156840 19362 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:09.157047 19362 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:09.162248 19367 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:09.162288 19370 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:09.162248 19368 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:09.162559 19362 server_base.cc:1034] running on GCE node
I20260809 02:15:09.162770 19362 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:09.163058 19362 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:09.164218 19362 hybrid_clock.cc:648] HybridClock initialized: now 1786241709164207 us; error 32 us; skew 500 ppm
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241709, etypes {rep=17 tkt=17 ses=17}, oryx/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:09.166925 19362 init.cc:377] Logged in from keytab as oryx/127.9.40.67@KRBTEST.COM (short username oryx)
I20260809 02:15:09.167382 19362 webserver.cc:460] Webserver started at http://127.9.40.67:44045/ using document root <none> and password file <none>
I20260809 02:15:09.167650 19362 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:09.167739 19362 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:09.169301 19362 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:15:09.169924 19378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.170185 19362 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:09.170300 19362 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "f3ce08be65a94ad89b8540dd8661c15a"
format_stamp: "Formatted at 2026-08-09 02:15:08 on dist-test-slave-qg60"
server_key: "ac7d9df3785bde28395a3a6aa1733ec0"
server_key_iv: "fe4ba2072c573ac596369aa6d4b19d1a"
server_key_version: "encryptionkey@0"
I20260809 02:15:09.170658 19362 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:09.183219 19362 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:09.185638 19362 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:09.185809 19362 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:09.186120 19362 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:09.186450 19362 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:09.186502 19362 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.186545 19362 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:09.186574 19362 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:09.192420 19362 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:38459
I20260809 02:15:09.192533 19499 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:38459 every 8 connection(s)
I20260809 02:15:09.192903 19362 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:09.194147 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19362
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.201535 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.193675 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:42495)
0809 02:15:09.193908 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.193959 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.194620 (+ 661us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.195075 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.195079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.195648 (+ 569us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.195914 (+ 266us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.195920 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.196674 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.196675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.197639 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.197642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.198142 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.198143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.198290 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.198818 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.198826 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.201383 (+ 2557us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
W20260809 02:15:09.201588 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.193688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:42495 (local address 127.9.40.126:33273)
0809 02:15:09.193761 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.193764 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.194069 (+ 305us) server_negotiation.cc:407] Connection header received
0809 02:15:09.194737 (+ 668us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.194738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.194749 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.194840 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.196029 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.196574 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.197756 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.198041 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.198120 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.201481 (+ 3361us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:42495: BlockingRecv error: recv got EOF from 127.9.40.67:42495 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:15:09.201979 19500 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:33273 (0 consecutive failures): Not authorized: Failed to ping master at 127.9.40.126:33273: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
I20260809 02:15:09.202802 19504 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.194718 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41546 (local address 127.9.40.126:33273)
0809 02:15:09.194854 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.194855 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.194935 (+ 80us) server_negotiation.cc:407] Connection header received
0809 02:15:09.194991 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.194991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.195002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.195070 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.195737 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.196348 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.197321 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.197642 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.197722 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.199098 (+ 1376us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:09.199103 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:09.199103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:09.199114 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:09.201202 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:09.201619 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:09.201622 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:09.201623 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:09.201740 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:09.202003 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:09.202004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:09.202004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:09.202286 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:09.202431 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:09.202580 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 02:15:09.202694 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":77,"threads_started":1}
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.206713 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.202242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:37707)
0809 02:15:09.202340 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.202353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.202397 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.202552 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.202553 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.202702 (+ 149us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.202762 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.202764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.203593 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.203594 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.204161 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.204162 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.204548 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.204549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.204626 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.205066 (+ 440us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.205071 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.206670 (+ 1599us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260809 02:15:09.206844 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.202290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37707 (local address 127.9.40.126:33273)
0809 02:15:09.202351 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.202354 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.202383 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:15:09.202461 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.202462 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.202473 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.202528 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.202827 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.203529 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.204258 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.204457 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.204546 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.206771 (+ 2225us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:37707: BlockingRecv error: recv got EOF from 127.9.40.67:37707 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.210747 19502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.207027 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:41577)
0809 02:15:09.207076 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.207087 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.207127 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.207304 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.207305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.207406 (+ 101us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.207430 (+ 24us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.207431 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.207981 (+ 550us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.207981 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.208451 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.208452 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.208797 (+ 345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.208798 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.208847 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.209171 (+ 324us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.209175 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.210705 (+ 1530us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:15:09.210868 19216 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.207144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41577 (local address 127.9.40.126:33273)
0809 02:15:09.207198 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.207200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.207208 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:15:09.207228 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.207229 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.207243 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.207291 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.207506 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.207920 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.208517 (+ 597us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.208688 (+ 171us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.208778 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.210799 (+ 2021us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:41577: BlockingRecv error: recv got EOF from 127.9.40.67:41577 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
W20260809 02:15:09.210880 19500 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:09 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:09.958361 19505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:09.953411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:45759)
0809 02:15:09.953612 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:09.953624 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:09.953729 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:09.953866 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:09.953867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:09.954039 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:09.954091 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.954092 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.954764 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.954766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.955338 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:09.955341 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.955746 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.955747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:09.955793 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.956316 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:09.956321 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:09.958243 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":126,"threads_started":1}
W20260809 02:15:09.958406 19506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:09.953512 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45759 (local address 127.9.40.126:33273)
0809 02:15:09.953638 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:09.953640 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:09.953716 (+ 76us) server_negotiation.cc:407] Connection header received
0809 02:15:09.953775 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:09.953776 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:09.953786 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:09.953852 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:09.954161 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.954672 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.955446 (+ 774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:09.955668 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:09.955731 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:09.958326 (+ 2595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:45759: BlockingRecv error: recv got EOF from 127.9.40.65:45759 (error 108)
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":53,"threads_started":1}
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:10.091930 19507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:10.086893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:51647)
0809 02:15:10.087163 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:10.087173 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:10.087257 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:10.087405 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:10.087406 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:10.087624 (+ 218us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:10.087675 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.087677 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.088407 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.088407 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.089005 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.089006 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.089395 (+ 389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.089396 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.089454 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.089953 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:10.089957 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:10.091818 (+ 1861us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":195,"threads_started":1}
W20260809 02:15:10.091981 19506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:10.086994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:51647 (local address 127.9.40.126:33273)
0809 02:15:10.087049 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:10.087056 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:10.087239 (+ 183us) server_negotiation.cc:407] Connection header received
0809 02:15:10.087315 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:10.087316 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:10.087329 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:10.087398 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:10.087762 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.088312 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.089108 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.089304 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.089395 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.091911 (+ 2516us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:51647: BlockingRecv error: recv got EOF from 127.9.40.66:51647 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:10.216464 19508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:10.211324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:38881)
0809 02:15:10.211547 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:10.211560 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:10.211636 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:10.211833 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:10.211834 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:10.212044 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:10.212118 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.212121 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.212979 (+ 858us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.212980 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.213615 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.213618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.213955 (+ 337us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.213956 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.213996 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.214550 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:10.214555 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:10.216359 (+ 1804us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":60,"threads_started":1}
W20260809 02:15:10.216606 19506 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:10.211496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:38881 (local address 127.9.40.126:33273)
0809 02:15:10.211550 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:10.211552 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:10.211700 (+ 148us) server_negotiation.cc:407] Connection header received
0809 02:15:10.211745 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:10.211746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:10.211763 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:10.211813 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:10.212217 (+ 404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.212919 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.213698 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.213886 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.213944 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.216495 (+ 2551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:38881: BlockingRecv error: recv got EOF from 127.9.40.67:38881 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:10 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
W20260809 02:15:10.964164 19511 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:10.958998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51541 (local address 127.9.40.126:33273)
0809 02:15:10.959163 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:10.959166 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:10.959175 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:10.959200 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:10.959201 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:10.959212 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:10.959277 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:10.959642 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.960267 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.961070 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.961270 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.961335 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.964095 (+ 2760us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:51541: BlockingRecv error: recv got EOF from 127.9.40.65:51541 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260809 02:15:10.964160 19510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:10.958927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:51541)
0809 02:15:10.959082 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:10.959093 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:10.959143 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:10.959286 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:10.959287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:10.959467 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:10.959519 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.959520 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.960365 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.960365 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.960977 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:10.960979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:10.961344 (+ 365us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:10.961345 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:10.961386 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:10.961898 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:10.961902 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:10.964060 (+ 2158us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":90,"threads_started":1}
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:11.098114 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:11.092541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:49907)
0809 02:15:11.092778 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:11.092792 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:11.092878 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:11.093123 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:11.093124 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:11.093343 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:11.093397 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.093399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.094176 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.094178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.094931 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.094935 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.095347 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.095348 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.095414 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.096064 (+ 650us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:11.096075 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:11.098047 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":114,"threads_started":1}
W20260809 02:15:11.098287 19511 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:11.092662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:49907 (local address 127.9.40.126:33273)
0809 02:15:11.092780 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:11.092782 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:11.092860 (+ 78us) server_negotiation.cc:407] Connection header received
0809 02:15:11.092955 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:11.092956 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:11.092968 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:11.093111 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:11.093507 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.094075 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.095029 (+ 954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.095259 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.095348 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.098222 (+ 2874us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:49907: BlockingRecv error: recv got EOF from 127.9.40.66:49907 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:11.222318 19513 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:11.217065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:33999)
0809 02:15:11.217195 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:11.217217 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:11.217278 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:11.217389 (+ 111us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:11.217390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:11.217575 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:11.217625 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.217627 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.218450 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.218451 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.219066 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.219068 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.219451 (+ 383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.219452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.219508 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.220053 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:11.220058 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:11.222252 (+ 2194us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":75,"threads_started":1}
W20260809 02:15:11.222445 19511 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:11.217163 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:33999 (local address 127.9.40.126:33273)
0809 02:15:11.217225 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:11.217229 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:11.217274 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:15:11.217320 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:11.217321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:11.217333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:11.217383 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:11.217772 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.218362 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.219162 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.219347 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.219413 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.222363 (+ 2950us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:33999: BlockingRecv error: recv got EOF from 127.9.40.67:33999 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:11 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:11.969831 19514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:11.964872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:44637)
0809 02:15:11.965019 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:11.965039 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:11.965133 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:11.965308 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:11.965309 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:11.965483 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:11.965533 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.965535 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.966183 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.966184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.966748 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:11.966750 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.967142 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.967143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:11.967206 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.967649 (+ 443us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:11.967654 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:11.969760 (+ 2106us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":88,"threads_started":1}
W20260809 02:15:11.969969 19515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:11.965038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44637 (local address 127.9.40.126:33273)
0809 02:15:11.965164 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:11.965166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:11.965182 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:15:11.965208 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:11.965209 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:11.965219 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:11.965288 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:11.965605 (+ 317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.966112 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.966828 (+ 716us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:11.967046 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:11.967105 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:11.969860 (+ 2755us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:44637: BlockingRecv error: recv got EOF from 127.9.40.65:44637 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":76,"threads_started":1}
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:12.104009 19517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:12.098775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:36419)
0809 02:15:12.098940 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:12.098954 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:12.099026 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:12.099202 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:12.099203 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:12.099405 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:12.099465 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.099467 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.100193 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.100194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.100829 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.100831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.101225 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.101226 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.101290 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.101821 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:12.101826 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:12.103931 (+ 2105us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":110,"threads_started":1}
W20260809 02:15:12.104188 19515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:12.098946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:36419 (local address 127.9.40.126:33273)
0809 02:15:12.099057 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:12.099059 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:12.099070 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:12.099106 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:12.099107 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:12.099123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:12.099191 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:12.099573 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.100097 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.100916 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.101129 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.101215 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.104064 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:36419: BlockingRecv error: recv got EOF from 127.9.40.66:36419 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:12.228659 19518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:12.223006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:48059)
0809 02:15:12.223211 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:12.223223 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:12.223344 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:12.223574 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:12.223576 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:12.223820 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:12.223885 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.223887 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.224893 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.224894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.225455 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.225457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.225886 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.225887 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.225932 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.226457 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:12.226462 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:12.228536 (+ 2074us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
W20260809 02:15:12.228704 19515 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:12.223097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48059 (local address 127.9.40.126:33273)
0809 02:15:12.223181 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:12.223183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:12.223352 (+ 169us) server_negotiation.cc:407] Connection header received
0809 02:15:12.223429 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:12.223430 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:12.223448 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:12.223517 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:12.224135 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.224804 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.225574 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.225800 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.225876 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.228637 (+ 2761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:48059: BlockingRecv error: recv got EOF from 127.9.40.67:48059 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:12 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:12.975679 19519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:12.970393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:36645)
0809 02:15:12.970659 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:12.970672 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:12.970747 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:12.970959 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:12.970960 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:12.971272 (+ 312us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:12.971334 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.971337 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.972064 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.972065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.972663 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:12.972664 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.973082 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.973083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:12.973142 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.973646 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:12.973650 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:12.975577 (+ 1927us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":90,"threads_started":1}
W20260809 02:15:12.975742 19520 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:12.970478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36645 (local address 127.9.40.126:33273)
0809 02:15:12.970656 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:12.970658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:12.970744 (+ 86us) server_negotiation.cc:407] Connection header received
0809 02:15:12.970822 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:12.970823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:12.970851 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:12.970933 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:12.971416 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.971968 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.972768 (+ 800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:12.972987 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:12.973080 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:12.975665 (+ 2585us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:36645: BlockingRecv error: recv got EOF from 127.9.40.65:36645 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:13.109335 19521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:13.104510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:56781)
0809 02:15:13.104690 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:13.104705 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:13.104827 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:13.104986 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:13.104987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:13.105210 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:13.105261 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.105264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.105974 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.105974 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.106501 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.106502 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.106855 (+ 353us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.106856 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.106913 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.107410 (+ 497us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:13.107414 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:13.109238 (+ 1824us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
W20260809 02:15:13.109391 19520 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:13.104606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56781 (local address 127.9.40.126:33273)
0809 02:15:13.104694 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:13.104697 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:13.104805 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:15:13.104893 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:13.104894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:13.104913 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:13.104980 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:13.105352 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.105874 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.106585 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.106779 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.106857 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.109326 (+ 2469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:56781: BlockingRecv error: recv got EOF from 127.9.40.66:56781 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:13.234546 19522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:13.229301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:36081)
0809 02:15:13.229621 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:13.229634 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:13.229735 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:13.229895 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:13.229895 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:13.230106 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:13.230156 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.230158 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.230863 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.230864 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.231392 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.231393 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.231762 (+ 369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.231763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.231822 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.232374 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:13.232379 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:13.234441 (+ 2062us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":137,"threads_started":1}
W20260809 02:15:13.234599 19520 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:13.229411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36081 (local address 127.9.40.126:33273)
0809 02:15:13.229612 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:13.229615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:13.229718 (+ 103us) server_negotiation.cc:407] Connection header received
0809 02:15:13.229795 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:13.229796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:13.229819 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:13.229889 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:13.230264 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.230767 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.231482 (+ 715us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.231675 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.231764 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.234527 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:36081: BlockingRecv error: recv got EOF from 127.9.40.67:36081 (error 108)
Metrics: {"server-negotiator.queue_time_us":167}
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:13 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:13.989259 19523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:13.976266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:39439)
0809 02:15:13.976469 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:13.976483 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:13.976554 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:13.976787 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:13.976789 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:13.977100 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:13.977154 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.977156 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.977842 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.977843 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.978534 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:13.978537 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.979006 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.979007 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:13.979055 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.986850 (+ 7795us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:13.986857 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:13.989193 (+ 2336us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":124,"threads_started":1}
W20260809 02:15:13.989488 19524 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:13.976375 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39439 (local address 127.9.40.126:33273)
0809 02:15:13.976566 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:13.976569 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:13.976605 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:15:13.976632 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:13.976633 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:13.976646 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:13.976766 (+ 120us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:13.977223 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.977742 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.978635 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:13.978856 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:13.978927 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:13.989383 (+ 10456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:39439: BlockingRecv error: recv got EOF from 127.9.40.65:39439 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:14.114872 19525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:14.109948 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:52941)
0809 02:15:14.110112 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:14.110124 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:14.110201 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:14.110350 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:14.110350 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:14.110543 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:14.110592 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.110594 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.111280 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.111281 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.111802 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.111803 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.112182 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.112183 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.112245 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.112799 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:14.112804 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:14.114769 (+ 1965us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":65,"threads_started":1}
W20260809 02:15:14.114921 19524 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:14.110134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52941 (local address 127.9.40.126:33273)
0809 02:15:14.110198 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:14.110200 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:14.110209 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:14.110273 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:14.110274 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:14.110287 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:14.110334 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:14.110673 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.111184 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.111886 (+ 702us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.112099 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.112184 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.114857 (+ 2673us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:52941: BlockingRecv error: recv got EOF from 127.9.40.66:52941 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:14.239847 19526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:14.235156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:43675)
0809 02:15:14.235340 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:14.235352 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:14.235415 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:14.235556 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:14.235557 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:14.235734 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:14.235785 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.235787 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.236486 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.236487 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.237064 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.237065 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.237384 (+ 319us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.237384 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.237431 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.237891 (+ 460us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:14.237896 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:14.239746 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":124,"threads_started":1}
W20260809 02:15:14.239894 19524 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:14.235201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43675 (local address 127.9.40.126:33273)
0809 02:15:14.235266 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:14.235268 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:14.235416 (+ 148us) server_negotiation.cc:407] Connection header received
0809 02:15:14.235474 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:14.235475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:14.235489 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:14.235538 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:14.235859 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.236405 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.237137 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.237314 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.237371 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.239827 (+ 2456us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:43675: BlockingRecv error: recv got EOF from 127.9.40.67:43675 (error 108)
Metrics: {"server-negotiator.queue_time_us":38}
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:14 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:14.994987 19527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:14.989777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:43011)
0809 02:15:14.989924 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:14.989944 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:14.989999 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:14.990260 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:14.990261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:14.990503 (+ 242us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:14.990595 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.990599 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.991266 (+ 667us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.991268 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.991892 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:14.991894 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.992287 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.992288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:14.992342 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.992926 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:14.992931 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:14.994916 (+ 1985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
W20260809 02:15:14.995159 19529 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:14.989890 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43011 (local address 127.9.40.126:33273)
0809 02:15:14.990059 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:14.990061 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:14.990077 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:15:14.990126 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:14.990127 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:14.990142 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:14.990229 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:14.990687 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.991189 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.991972 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:14.992177 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:14.992237 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:14.995038 (+ 2801us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:43011: BlockingRecv error: recv got EOF from 127.9.40.65:43011 (error 108)
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":111,"threads_started":1}
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:15.120229 19530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:15.115406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:43393)
0809 02:15:15.115609 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:15.115620 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:15.115686 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:15.115817 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:15.115818 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:15.115991 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:15.116082 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.116084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.116720 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.116721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.117383 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.117386 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.117741 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.117742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.117783 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:15.118343 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:15.118347 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:15.120169 (+ 1822us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":138,"threads_started":1}
W20260809 02:15:15.120424 19529 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:15.115471 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43393 (local address 127.9.40.126:33273)
0809 02:15:15.115557 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:15.115559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:15.115683 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:15:15.115735 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:15.115736 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:15.115747 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:15.115798 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:15.116145 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.116653 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.117458 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.117667 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.117723 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:15.120348 (+ 2625us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:43393: BlockingRecv error: recv got EOF from 127.9.40.66:43393 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:15.245803 19531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:15.240381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:37233)
0809 02:15:15.240594 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:15.240614 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:15.240691 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:15.240901 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:15.240902 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:15.241098 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:15.241152 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.241154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.241864 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.241865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.242477 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.242479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.242823 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.242823 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.242872 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:15.243463 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:15.243469 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:15.245706 (+ 2237us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":67,"threads_started":1}
W20260809 02:15:15.245843 19529 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:15.240534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37233 (local address 127.9.40.126:33273)
0809 02:15:15.240610 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:15.240612 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:15.240681 (+ 69us) server_negotiation.cc:407] Connection header received
0809 02:15:15.240802 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:15.240803 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:15.240817 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:15.240868 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:15.241249 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.241783 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.242564 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.242748 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.242802 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:15.245781 (+ 2979us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:37233: BlockingRecv error: recv got EOF from 127.9.40.67:37233 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:15 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:16.000439 19532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:15.995484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:56789)
0809 02:15:15.995701 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:15.995712 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:15.995769 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:15.995923 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:15.995924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:15.996124 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:15.996176 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.996178 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.996905 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.996906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.997466 (+ 560us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:15.997468 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.997849 (+ 381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.997851 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:15.997901 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:15.998395 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:15.998400 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:16.000375 (+ 1975us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
W20260809 02:15:16.000646 19533 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:15.995618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56789 (local address 127.9.40.126:33273)
0809 02:15:15.995753 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:15.995755 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:15.995764 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:15.995819 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:15.995819 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:15.995829 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:15.995889 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:15.996280 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.996827 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.997553 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:15.997751 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:15.997811 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:16.000531 (+ 2720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:56789: BlockingRecv error: recv got EOF from 127.9.40.65:56789 (error 108)
Metrics: {"server-negotiator.queue_time_us":108,"thread_start_us":76,"threads_started":1}
Aug 09 02:15:16 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:16 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:16.125849 19534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:16.120827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:55019)
0809 02:15:16.121021 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:16.121032 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:16.121097 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:16.121232 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:16.121232 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:16.121402 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:16.121455 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:16.121457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.122130 (+ 673us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.122131 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:16.122679 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:16.122681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.123078 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.123079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:16.123133 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:16.123656 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:16.123661 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:16.125761 (+ 2100us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":114,"threads_started":1}
W20260809 02:15:16.125965 19533 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:16.120914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:55019 (local address 127.9.40.126:33273)
0809 02:15:16.120994 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:16.120996 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:16.121096 (+ 100us) server_negotiation.cc:407] Connection header received
0809 02:15:16.121145 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:16.121147 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:16.121158 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:16.121210 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:16.121546 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.122041 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.122763 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.122953 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.123016 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:16.125869 (+ 2853us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:55019: BlockingRecv error: recv got EOF from 127.9.40.66:55019 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 02:15:16 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:16 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:16.252005 19535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:16.246377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:52493)
0809 02:15:16.246560 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:16.246594 (+ 34us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:16.246672 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:16.246831 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:16.246832 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:16.247019 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:16.247069 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:16.247071 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.247764 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.247766 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:16.248314 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:16.248316 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.248706 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.248707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:16.248791 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:16.249359 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:16.249372 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:16.251946 (+ 2574us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":114,"threads_started":1}
W20260809 02:15:16.252126 19533 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:16.246547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:52493 (local address 127.9.40.126:33273)
0809 02:15:16.246623 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:16.246625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:16.246652 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:15:16.246756 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:16.246757 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:16.246772 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:16.246826 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:16.247155 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.247668 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.248411 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:16.248617 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:16.248694 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:16.252053 (+ 3359us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:52493: BlockingRecv error: recv got EOF from 127.9.40.67:52493 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:17.006059 19537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:17.001141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:48965)
0809 02:15:17.001342 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:17.001358 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:17.001430 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:17.001623 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:17.001624 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:17.001797 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:17.001849 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.001850 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.002526 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.002527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.003056 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.003058 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.003417 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.003418 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.003474 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.003973 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:17.003978 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:17.005952 (+ 1974us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":82,"threads_started":1}
W20260809 02:15:17.006150 19539 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:17.001298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:48965 (local address 127.9.40.126:33273)
0809 02:15:17.001464 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:17.001465 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:17.001474 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:15:17.001499 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:17.001499 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:17.001531 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:17.001597 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:17.001927 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.002444 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.003132 (+ 688us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.003320 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.003373 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.006059 (+ 2686us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:48965: BlockingRecv error: recv got EOF from 127.9.40.65:48965 (error 108)
Metrics: {"server-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:17.131479 19540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:17.126542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:54243)
0809 02:15:17.126721 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:17.126741 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:17.126802 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:17.126931 (+ 129us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:17.126932 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:17.127114 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:17.127164 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.127166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.127987 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.127988 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.128545 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.128546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.128892 (+ 346us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.128892 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.128949 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.129533 (+ 584us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:17.129538 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:17.131417 (+ 1879us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":89,"threads_started":1}
W20260809 02:15:17.131603 19539 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:17.126656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:54243 (local address 127.9.40.126:33273)
0809 02:15:17.126724 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:17.126726 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:17.126801 (+ 75us) server_negotiation.cc:407] Connection header received
0809 02:15:17.126847 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:17.126847 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:17.126859 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:17.126906 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:17.127238 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.127893 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.128625 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.128830 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.128883 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.131552 (+ 2669us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:54243: BlockingRecv error: recv got EOF from 127.9.40.66:54243 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:17 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:17.257817 19541 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:17.252578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:48591)
0809 02:15:17.252724 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:17.252788 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:17.252856 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:17.253016 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:17.253017 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:17.253231 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:17.253281 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.253283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.253965 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.253966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.254515 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:17.254517 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.254910 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.254911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:17.254977 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.255535 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:17.255540 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:17.257714 (+ 2174us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
W20260809 02:15:17.257869 19539 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:17.252777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48591 (local address 127.9.40.126:33273)
0809 02:15:17.252840 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:17.252843 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:17.252854 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:17.252921 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:17.252922 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:17.252942 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:17.253060 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:17.253368 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.253876 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.254611 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:17.254801 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:17.254908 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:17.257800 (+ 2892us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:48591: BlockingRecv error: recv got EOF from 127.9.40.67:48591 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:18.012223 19542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:18.006624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:40711)
0809 02:15:18.006828 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:18.006841 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:18.006903 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:18.007046 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:18.007047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:18.007233 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:18.007285 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.007286 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.008173 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.008174 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.008723 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.008725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.009234 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.009235 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.009279 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.009742 (+ 463us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:18.009746 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:18.012098 (+ 2352us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":116,"threads_started":1}
W20260809 02:15:18.012328 19543 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:18.006750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40711 (local address 127.9.40.126:33273)
0809 02:15:18.006920 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:18.006922 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:18.006933 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:18.006960 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:18.006961 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:18.006972 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:18.007034 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:18.007396 (+ 362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.008072 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.008880 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.009157 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.009218 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.012228 (+ 3010us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:40711: BlockingRecv error: recv got EOF from 127.9.40.65:40711 (error 108)
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":61,"threads_started":1}
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:18.138092 19544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:18.132281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:55647)
0809 02:15:18.132492 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:18.132505 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:18.132657 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:18.132949 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:18.132950 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:18.133224 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:18.133292 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.133294 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.134177 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.134178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.134766 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.134768 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.135195 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.135196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.135262 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.135806 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:18.135811 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:18.138019 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":125,"threads_started":1}
W20260809 02:15:18.138285 19543 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:18.132356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:55647 (local address 127.9.40.126:33273)
0809 02:15:18.132432 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:18.132435 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:18.132648 (+ 213us) server_negotiation.cc:407] Connection header received
0809 02:15:18.132777 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:18.132778 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:18.132795 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:18.132857 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:18.133407 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.134011 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.134863 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.135093 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.135193 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.138210 (+ 3017us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:55647: BlockingRecv error: recv got EOF from 127.9.40.66:55647 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:18 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:18.263432 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:18.258414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:58061)
0809 02:15:18.258629 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:18.258644 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:18.258731 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:18.258921 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:18.258922 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:18.259135 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:18.259186 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.259188 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.259887 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.259888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.260413 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:18.260414 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.260879 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.260880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:18.260949 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.261449 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:18.261454 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:18.263369 (+ 1915us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":104,"threads_started":1}
W20260809 02:15:18.263638 19543 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:18.258504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58061 (local address 127.9.40.126:33273)
0809 02:15:18.258602 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:18.258605 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:18.258708 (+ 103us) server_negotiation.cc:407] Connection header received
0809 02:15:18.258813 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:18.258814 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:18.258839 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:18.258911 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:18.259290 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.259796 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.260515 (+ 719us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:18.260801 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:18.260880 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:18.263565 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:58061: BlockingRecv error: recv got EOF from 127.9.40.67:58061 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:19.021503 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:19.012787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:51323)
0809 02:15:19.013006 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:19.013019 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:19.013116 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:19.013331 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:19.013332 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:19.013522 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:19.013573 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.013575 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.014378 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.014379 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.014985 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.014987 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.015420 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.015421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.015475 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.019085 (+ 3610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:19.019091 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:19.021437 (+ 2346us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":66,"threads_started":1}
W20260809 02:15:19.021701 19547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:19.012883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:51323 (local address 127.9.40.126:33273)
0809 02:15:19.013037 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:19.013040 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:19.013073 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:15:19.013238 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:19.013239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:19.013251 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:19.013303 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:19.013645 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.014237 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.015081 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.015279 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.015337 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.021599 (+ 6262us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:51323: BlockingRecv error: recv got EOF from 127.9.40.65:51323 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":74,"threads_started":1}
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:19.143487 19549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:19.138676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:42847)
0809 02:15:19.138851 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:19.138863 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:19.138931 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:19.139119 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:19.139120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:19.139292 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:19.139348 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.139349 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.140030 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.140030 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.140559 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.140560 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.141037 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.141038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.141083 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.141534 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:19.141539 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:19.143402 (+ 1863us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":111,"threads_started":1}
W20260809 02:15:19.143673 19547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:19.138745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42847 (local address 127.9.40.126:33273)
0809 02:15:19.138823 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:19.138826 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:19.138931 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:15:19.138983 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:19.138984 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:19.138996 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:19.139062 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:19.139410 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.139957 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.140648 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.140918 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.140984 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.143549 (+ 2565us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:42847: BlockingRecv error: recv got EOF from 127.9.40.66:42847 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:19 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:19.269170 19550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:19.264018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:56191)
0809 02:15:19.264277 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:19.264294 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:19.264373 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:19.264543 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:19.264543 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:19.264773 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:19.264839 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.264842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.265653 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.265654 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.266199 (+ 545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:19.266201 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.266577 (+ 376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.266578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:19.266646 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.267168 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:19.267173 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:19.269069 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":193,"threads_started":1}
W20260809 02:15:19.269212 19547 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:19.264171 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56191 (local address 127.9.40.126:33273)
0809 02:15:19.264245 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:19.264247 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:19.264416 (+ 169us) server_negotiation.cc:407] Connection header received
0809 02:15:19.264449 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:19.264450 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:19.264467 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:19.264516 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:19.264939 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.265561 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.266293 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:19.266497 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:19.266577 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:19.269148 (+ 2571us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:56191: BlockingRecv error: recv got EOF from 127.9.40.67:56191 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:20.026891 19552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:20.021965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:36079)
0809 02:15:20.022317 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:20.022327 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:20.022380 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:20.022584 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:20.022585 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:20.022756 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:20.022804 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.022805 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.023530 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.023531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.024103 (+ 572us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.024106 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.024440 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.024441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.024480 (+ 39us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.025083 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:20.025092 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:20.026818 (+ 1726us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":248,"threads_started":1}
W20260809 02:15:20.027148 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:20.022142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36079 (local address 127.9.40.126:33273)
0809 02:15:20.022404 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:20.022406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:20.022416 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:20.022444 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:20.022445 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:20.022460 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:20.022541 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:20.022878 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.023441 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.024184 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.024372 (+ 188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.024424 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.027076 (+ 2652us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:36079: BlockingRecv error: recv got EOF from 127.9.40.65:36079 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":68,"threads_started":1}
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:20.148873 19554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:20.144052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:53311)
0809 02:15:20.144231 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:20.144253 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:20.144324 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:20.144480 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:20.144480 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:20.144658 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:20.144707 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.144709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.145398 (+ 689us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.145398 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.145934 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.145936 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.146339 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.146340 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.146393 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.146974 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:20.146979 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:20.148799 (+ 1820us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
W20260809 02:15:20.148929 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:20.144184 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:53311 (local address 127.9.40.126:33273)
0809 02:15:20.144236 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:20.144237 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:20.144334 (+ 97us) server_negotiation.cc:407] Connection header received
0809 02:15:20.144375 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:20.144376 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:20.144386 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:20.144435 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:20.144819 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.145323 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.146054 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.146244 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.146301 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.148871 (+ 2570us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:53311: BlockingRecv error: recv got EOF from 127.9.40.66:53311 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:20 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:20.274636 19555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:20.269687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:59821)
0809 02:15:20.269871 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:20.269880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:20.269940 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:20.270112 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:20.270120 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:20.270297 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:20.270346 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.270348 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.270992 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.270993 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.271546 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:20.271547 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.271874 (+ 327us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.271875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:20.271918 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.272517 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:20.272522 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:20.274523 (+ 2001us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":103,"threads_started":1}
W20260809 02:15:20.274760 19553 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:20.269770 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59821 (local address 127.9.40.126:33273)
0809 02:15:20.269834 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:20.269836 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:20.269937 (+ 101us) server_negotiation.cc:407] Connection header received
0809 02:15:20.269988 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:20.269989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:20.270003 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:20.270063 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:20.270416 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.270924 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.271622 (+ 698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:20.271805 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:20.271859 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:20.274651 (+ 2792us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:59821: BlockingRecv error: recv got EOF from 127.9.40.67:59821 (error 108)
Metrics: {"server-negotiator.queue_time_us":34}
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:21.032641 19556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:21.027407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:35567)
0809 02:15:21.027571 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:21.027586 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:21.027664 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:21.027813 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:21.027814 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:21.027987 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:21.028049 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.028051 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.028711 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.028713 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.029384 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.029386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.029766 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.029767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.029815 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.030440 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:21.030447 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:21.032519 (+ 2072us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":76,"threads_started":1}
W20260809 02:15:21.032689 19558 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:21.027481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:35567 (local address 127.9.40.126:33273)
0809 02:15:21.027651 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:21.027653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:21.027663 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:21.027727 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:21.027727 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:21.027738 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:21.027790 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:21.028113 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.028616 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.029476 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.029671 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.029732 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.032611 (+ 2879us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:35567: BlockingRecv error: recv got EOF from 127.9.40.65:35567 (error 108)
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":78,"threads_started":1}
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:21.154465 19559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:21.149422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:57213)
0809 02:15:21.149608 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:21.149621 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:21.149725 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:21.149922 (+ 197us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:21.149923 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:21.150136 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:21.150184 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.150186 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.150964 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.150965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.151550 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.151551 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.151899 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.151900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.151941 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.152506 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:21.152510 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:21.154348 (+ 1838us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":93,"threads_started":1}
W20260809 02:15:21.154506 19558 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:21.149533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57213 (local address 127.9.40.126:33273)
0809 02:15:21.149631 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:21.149634 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:21.149705 (+ 71us) server_negotiation.cc:407] Connection header received
0809 02:15:21.149838 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:21.149839 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:21.149851 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:21.149910 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:21.150270 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.150836 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.151619 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.151824 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.151882 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.154438 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:57213: BlockingRecv error: recv got EOF from 127.9.40.66:57213 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:21 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:21.279959 19560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:21.275076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:59631)
0809 02:15:21.275245 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:21.275258 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:21.275350 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:21.275513 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:21.275514 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:21.275696 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:21.275746 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.275748 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.276491 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.276492 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.277120 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:21.277123 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.277457 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.277458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:21.277518 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.278018 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:21.278023 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:21.279898 (+ 1875us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":66,"threads_started":1}
W20260809 02:15:21.280061 19558 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:21.275214 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59631 (local address 127.9.40.126:33273)
0809 02:15:21.275290 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:21.275292 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:21.275332 (+ 40us) server_negotiation.cc:407] Connection header received
0809 02:15:21.275405 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:21.275406 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:21.275419 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:21.275467 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:21.275813 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.276400 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.277203 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:21.277386 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:21.277443 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:21.279999 (+ 2556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:59631: BlockingRecv error: recv got EOF from 127.9.40.67:59631 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:22.038864 19561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:22.033278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:32999)
0809 02:15:22.033503 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:22.033516 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:22.033579 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:22.033726 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:22.033726 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:22.033903 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:22.033955 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.033956 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.034716 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.034717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.035269 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.035272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.035704 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.035705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.035758 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.036375 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:22.036380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:22.038746 (+ 2366us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":118,"threads_started":1}
W20260809 02:15:22.038916 19562 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:22.033417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:32999 (local address 127.9.40.126:33273)
0809 02:15:22.033589 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:22.033591 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:22.033601 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:22.033629 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:22.033630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:22.033641 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:22.033707 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:22.034037 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.034621 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.035380 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.035606 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.035675 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.038835 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:32999: BlockingRecv error: recv got EOF from 127.9.40.65:32999 (error 108)
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:22.159785 19564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:22.155025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:51591)
0809 02:15:22.155189 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:22.155200 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:22.155251 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:22.155401 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:22.155401 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:22.155588 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:22.155638 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.155640 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.156348 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.156349 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.156942 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.156944 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.157315 (+ 371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.157316 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.157367 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.157834 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:22.157839 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:22.159712 (+ 1873us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":98,"threads_started":1}
W20260809 02:15:22.159970 19562 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:22.155176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:51591 (local address 127.9.40.126:33273)
0809 02:15:22.155242 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:22.155244 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:22.155255 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:22.155312 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:22.155312 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:22.155323 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:22.155380 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:22.155748 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.156263 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.157015 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.157217 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.157277 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.159859 (+ 2582us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:51591: BlockingRecv error: recv got EOF from 127.9.40.66:51591 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:22 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:22.285833 19565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:22.280547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:39843)
0809 02:15:22.280762 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:22.280771 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:22.280849 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:22.281011 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:22.281011 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:22.281254 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:22.281306 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.281308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.282094 (+ 786us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.282096 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.282872 (+ 776us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:22.282874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.283218 (+ 344us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.283219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:22.283262 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.283742 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:22.283747 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:22.285762 (+ 2015us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":101,"threads_started":1}
W20260809 02:15:22.285939 19562 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:22.280614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:39843 (local address 127.9.40.126:33273)
0809 02:15:22.280687 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:22.280689 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:22.280834 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:15:22.280911 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:22.280912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:22.280928 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:22.281004 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:22.281399 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.281963 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.282944 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:22.283143 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:22.283201 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:22.285864 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:39843: BlockingRecv error: recv got EOF from 127.9.40.67:39843 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:23.045499 19566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:23.039449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:43279)
0809 02:15:23.039652 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:23.039665 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:23.039738 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:23.039904 (+ 166us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:23.039906 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:23.040085 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:23.040136 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.040138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.041027 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.041029 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.041884 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.041888 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.042430 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.042431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.042491 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.043231 (+ 740us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:23.043237 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:23.045433 (+ 2196us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":178,"thread_start_us":88,"threads_started":1}
W20260809 02:15:23.045714 19568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:23.039552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:43279 (local address 127.9.40.126:33273)
0809 02:15:23.039717 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:23.039719 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:23.039729 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:23.039806 (+ 77us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:23.039806 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:23.039817 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:23.039878 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:23.040231 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.040941 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.042011 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.042363 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.042430 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.045566 (+ 3136us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:43279: BlockingRecv error: recv got EOF from 127.9.40.65:43279 (error 108)
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":88,"threads_started":1}
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:23.165755 19569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:23.160471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:57227)
0809 02:15:23.160673 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:23.160687 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:23.160788 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:23.160936 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:23.160937 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:23.161138 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:23.161189 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.161191 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.161865 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.161867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.162726 (+ 859us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.162730 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.163146 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.163147 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.163198 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.163751 (+ 553us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:23.163756 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:23.165676 (+ 1920us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":98,"threads_started":1}
W20260809 02:15:23.165904 19568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:23.160559 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57227 (local address 127.9.40.126:33273)
0809 02:15:23.160668 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:23.160670 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:23.160778 (+ 108us) server_negotiation.cc:407] Connection header received
0809 02:15:23.160843 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:23.160844 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:23.160855 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:23.160908 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:23.161266 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.161773 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.162816 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.163058 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.163115 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.165819 (+ 2704us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:57227: BlockingRecv error: recv got EOF from 127.9.40.66:57227 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:23 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.67@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:23.291121 19570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:23.286362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.67:59109)
0809 02:15:23.286581 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:23.286595 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:23.286655 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:23.286798 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:23.286799 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:23.287028 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:23.287081 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.287083 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.287723 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.287723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.288296 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:23.288298 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.288610 (+ 312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.288611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:23.288651 (+ 40us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.289228 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:23.289233 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:23.291016 (+ 1783us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":113,"threads_started":1}
W20260809 02:15:23.291215 19568 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:23.286545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:59109 (local address 127.9.40.126:33273)
0809 02:15:23.286629 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:23.286631 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:23.286657 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:15:23.286720 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:23.286721 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:23.286733 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:23.286775 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:23.287159 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.287656 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.288370 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:23.288548 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:23.288599 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:23.291149 (+ 2550us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.67:59109: BlockingRecv error: recv got EOF from 127.9.40.67:59109 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 09 02:15:24 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:24 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.65@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:24.057119 19571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.046083 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.65:55463)
0809 02:15:24.046219 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:24.046241 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:24.046316 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:24.046525 (+ 209us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:24.046526 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:24.046750 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:24.046812 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.046814 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.047878 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.047879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.048509 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.048511 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.048964 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.048965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.049007 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.054944 (+ 5937us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:24.054950 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:24.057015 (+ 2065us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":77,"threads_started":1}
W20260809 02:15:24.057228 19572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:24.046211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:55463 (local address 127.9.40.126:33273)
0809 02:15:24.046364 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.046366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.046378 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:15:24.046408 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.046409 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.046423 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.046509 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.046892 (+ 383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.047771 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.048597 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.048887 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.048965 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.057128 (+ 8163us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.65:55463: BlockingRecv error: recv got EOF from 127.9.40.65:55463 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":70,"threads_started":1}
Aug 09 02:15:24 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
Aug 09 02:15:24 dist-test-slave-qg60 krb5kdc[18433](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.9.40.66@KRBTEST.COM for oryx/127.9.40.126@KRBTEST.COM, Server not found in Kerberos database
I20260809 02:15:24.171325 19573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.166522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33273 (local address 127.9.40.66:33327)
0809 02:15:24.166665 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:24.166680 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:24.166747 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:24.166886 (+ 139us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:24.166887 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:24.167070 (+ 183us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:24.167121 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.167123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.167810 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.167811 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.168343 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.168344 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.168723 (+ 379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.168723 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.168830 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.169381 (+ 551us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:24.169385 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:24.171216 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.9.40.126:33273: Server oryx/127.9.40.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":80,"threads_started":1}
W20260809 02:15:24.171378 19572 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:24.166649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:33327 (local address 127.9.40.126:33273)
0809 02:15:24.166729 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.166732 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.166742 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:24.166810 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.166812 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.166824 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.166870 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.167206 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.167721 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.168435 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.168644 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.168725 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.171313 (+ 2588us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.9.40.66:33327: BlockingRecv error: recv got EOF from 127.9.40.66:33327 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
I20260809 02:15:24.204810 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19054
I20260809 02:15:24.212437 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19217
I20260809 02:15:24.218963 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19362
I20260809 02:15:24.225800 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 18981
2026-08-09T02:15:24Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17643 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-09T02:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:24Z Disabled control of system clock
I20260809 02:15:24.257033 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33937
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:33937
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260809 02:15:24.339038 19577 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:24.339258 19577 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:24.339277 19577 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:24.340363 19577 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:24.340399 19577 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:24.340412 19577 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:24.340425 19577 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:24.340437 19577 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:24.341764 19577 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:33937
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:33937
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19577
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:24.341962 19577 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:24.342201 19577 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:24.348274 19583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.348265 19582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.348269 19585 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.348344 19577 server_base.cc:1034] running on GCE node
I20260809 02:15:24.348819 19577 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:24.349324 19577 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:24.350591 19577 hybrid_clock.cc:648] HybridClock initialized: now 1786241724350561 us; error 42 us; skew 500 ppm
I20260809 02:15:24.351840 19577 webserver.cc:460] Webserver started at http://127.9.40.126:46047/ using document root <none> and password file <none>
I20260809 02:15:24.352121 19577 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:24.352198 19577 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:24.352351 19577 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:24.353485 19577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "2ad193f8220b47c7870053fa16feffc6"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "3921e5745bb0578cdb421d5f8c97dd5b"
server_key_iv: "33024478832f708e2ec0870ef4a1c043"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.353861 19577 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2ad193f8220b47c7870053fa16feffc6"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "3921e5745bb0578cdb421d5f8c97dd5b"
server_key_iv: "33024478832f708e2ec0870ef4a1c043"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.355196 19577 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:15:24.356040 19591 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.356243 19577 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:24.356353 19577 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "2ad193f8220b47c7870053fa16feffc6"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "3921e5745bb0578cdb421d5f8c97dd5b"
server_key_iv: "33024478832f708e2ec0870ef4a1c043"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.356433 19577 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:24.372822 19577 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:24.373176 19577 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:24.373338 19577 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:24.377472 19577 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:33937
I20260809 02:15:24.377509 19646 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:33937 every 8 connection(s)
I20260809 02:15:24.377869 19577 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:24.378610 19647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:24.381517 19647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: Bootstrap starting.
I20260809 02:15:24.382059 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19577
I20260809 02:15:24.382189 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:15:24.382223 19647 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:24.382419 9377 external_mini_cluster.cc:1428] Setting key 130bcf5e719a7da6f1683775a6bdf771
I20260809 02:15:24.382532 19647 log.cc:824] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:24.383360 19647 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: No bootstrap required, opened a new log
I20260809 02:15:24.385351 19647 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } }
I20260809 02:15:24.385530 19647 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:24.385572 19647 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ad193f8220b47c7870053fa16feffc6, State: Initialized, Role: FOLLOWER
I20260809 02:15:24.385704 19647 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } }
I20260809 02:15:24.386041 19647 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:24.386083 19647 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:24.386125 19647 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:24.386811 19647 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } }
I20260809 02:15:24.386901 19647 leader_election.cc:302] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ad193f8220b47c7870053fa16feffc6; no voters:
I20260809 02:15:24.387149 19647 leader_election.cc:288] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:24.387202 19652 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:24.387475 19652 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [term 1 LEADER]: Becoming Leader. State: Replica: 2ad193f8220b47c7870053fa16feffc6, State: Running, Role: LEADER
I20260809 02:15:24.387565 19647 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:24.387610 19652 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } }
I20260809 02:15:24.388017 19655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ad193f8220b47c7870053fa16feffc6. Latest consensus state: current_term: 1 leader_uuid: "2ad193f8220b47c7870053fa16feffc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } } }
I20260809 02:15:24.388211 19655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:24.388746 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.383039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58334 (local address 127.9.40.126:33937)
0809 02:15:24.383261 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.383269 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.383296 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:15:24.383902 (+ 606us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.383912 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.383978 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.384107 (+ 129us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.384382 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.385430 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.386565 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.387056 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.387178 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.387949 (+ 771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:24.387953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:24.387953 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:24.388176 (+ 223us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:24.388219 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:24.388235 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:24.388303 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:24.388425 (+ 122us) server_negotiation.cc:299] Negotiation successful
0809 02:15:24.388507 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":100,"threads_started":1}
I20260809 02:15:24.388346 19654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ad193f8220b47c7870053fa16feffc6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ad193f8220b47c7870053fa16feffc6" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 33937 } } }
I20260809 02:15:24.388927 19654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6 [sys.catalog]: This master's current role is: LEADER
W20260809 02:15:24.389611 19670 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:15:24.389668 19670 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:15:24.389715 19663 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:24.389854 19663 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:24.391376 19663 catalog_manager.cc:1347] Generated new cluster ID: 07361e9c57d7494b94e9eb4aadb0006f
I20260809 02:15:24.391434 19663 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:24.404556 19663 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:24.405297 19663 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:24.411595 19663 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ad193f8220b47c7870053fa16feffc6: Generated new TSK 0
I20260809 02:15:24.411806 19663 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:15:24.415275 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--builtin_ntp_servers=127.9.40.84:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 02:15:24.505945 19673 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:24.506187 19673 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:24.506213 19673 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:24.507541 19673 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:24.507588 19673 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:24.507618 19673 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:24.509274 19673 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19673
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:24.509569 19673 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:24.509922 19673 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:24.516469 19683 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.516463 19680 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.516768 19673 server_base.cc:1034] running on GCE node
W20260809 02:15:24.516479 19679 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.517233 19673 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:24.517527 19673 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:24.518709 19673 hybrid_clock.cc:648] HybridClock initialized: now 1786241724518686 us; error 38 us; skew 500 ppm
I20260809 02:15:24.520210 19673 webserver.cc:460] Webserver started at http://127.9.40.65:45729/ using document root <none> and password file <none>
I20260809 02:15:24.520509 19673 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:24.520581 19673 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:24.520780 19673 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:24.521960 19673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "ce42f7abfa524fa2bd5f8c21b96a5b9c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "143be45724b6f44cc62906742f8f69ed"
server_key_iv: "bfd17d1f895a55427ff3dde069d3af14"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.522325 19673 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "ce42f7abfa524fa2bd5f8c21b96a5b9c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "143be45724b6f44cc62906742f8f69ed"
server_key_iv: "bfd17d1f895a55427ff3dde069d3af14"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.523741 19673 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:15:24.524581 19690 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.524796 19673 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:24.524899 19673 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "ce42f7abfa524fa2bd5f8c21b96a5b9c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "143be45724b6f44cc62906742f8f69ed"
server_key_iv: "bfd17d1f895a55427ff3dde069d3af14"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.525053 19673 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:24.533849 19673 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:24.534174 19673 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:24.534329 19673 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:24.534564 19673 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:24.534904 19673 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:24.534969 19673 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.535040 19673 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:24.535070 19673 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.541244 19673 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:41243
I20260809 02:15:24.541325 19812 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:41243 every 8 connection(s)
I20260809 02:15:24.541656 19673 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:24.547086 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.542466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:48287 (local address 127.9.40.126:33937)
0809 02:15:24.542557 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.542559 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.542768 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:15:24.543265 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.543265 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.543279 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.543352 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.544124 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.544622 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.545698 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.545887 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.545959 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.546788 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:24.546789 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:24.546790 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:24.546794 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:24.546807 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:24.546811 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:24.546856 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:24.546940 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:15:24.547024 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:15:24.547243 19815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.542426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33937 (local address 127.9.40.65:48287)
0809 02:15:24.542638 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:24.542679 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:24.543180 (+ 501us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:24.543404 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:24.543407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:24.543415 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:24.544015 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.544021 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.544707 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.544707 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.545614 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.545616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.545967 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.545968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.546109 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.546659 (+ 550us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:24.546659 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:24.546687 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:24.546708 (+ 21us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:24.546866 (+ 158us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:24.546870 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:24.546872 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:15:24.546926 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:15:24.547005 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260809 02:15:24.547811 19813 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33937
I20260809 02:15:24.547933 19813 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:24.548123 19813 heartbeater.cc:511] Master 127.9.40.126:33937 requested a full tablet report, sending...
I20260809 02:15:24.548663 19609 ts_manager.cc:194] Registered new tserver with Master: ce42f7abfa524fa2bd5f8c21b96a5b9c (127.9.40.65:41243)
I20260809 02:15:24.549363 19609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:48287
I20260809 02:15:24.550940 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19673
I20260809 02:15:24.551015 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:24.551227 9377 external_mini_cluster.cc:1428] Setting key 3e11ce7d0e9cde66ec032c5e05a543c7
I20260809 02:15:24.552160 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--builtin_ntp_servers=127.9.40.84:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260809 02:15:24.636267 19816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:24.636494 19816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:24.636514 19816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:24.637665 19816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:24.637703 19816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:24.637728 19816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:24.639001 19816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:24.639191 19816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:24.639446 19816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:24.644421 19823 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.644418 19822 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.644418 19826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.644654 19816 server_base.cc:1034] running on GCE node
I20260809 02:15:24.644907 19816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:24.645221 19816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:24.646401 19816 hybrid_clock.cc:648] HybridClock initialized: now 1786241724646387 us; error 28 us; skew 500 ppm
I20260809 02:15:24.647768 19816 webserver.cc:460] Webserver started at http://127.9.40.66:42371/ using document root <none> and password file <none>
I20260809 02:15:24.648034 19816 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:24.648139 19816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:24.648301 19816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:24.649473 19816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "9a802c55eeab4f2faefd5ddaf369c564"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "8e30d5200388b77537c8f57ba027b9e8"
server_key_iv: "4a8ae2ff6db5367c00930ee0a3692395"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.649850 19816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "9a802c55eeab4f2faefd5ddaf369c564"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "8e30d5200388b77537c8f57ba027b9e8"
server_key_iv: "4a8ae2ff6db5367c00930ee0a3692395"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.651234 19816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:15:24.652037 19832 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.652230 19816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:15:24.652334 19816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "9a802c55eeab4f2faefd5ddaf369c564"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "8e30d5200388b77537c8f57ba027b9e8"
server_key_iv: "4a8ae2ff6db5367c00930ee0a3692395"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.652410 19816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:24.677181 19816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:24.677546 19816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:24.677711 19816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:24.677986 19816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:24.678354 19816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:24.678409 19816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.678452 19816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:24.678494 19816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.684499 19816 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:37139
I20260809 02:15:24.684559 19951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:37139 every 8 connection(s)
I20260809 02:15:24.684888 19816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:24.687780 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19816
I20260809 02:15:24.687875 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:24.688107 9377 external_mini_cluster.cc:1428] Setting key a41aff0a29a29d5f1de2df518a0d93c2
I20260809 02:15:24.689111 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--builtin_ntp_servers=127.9.40.84:43581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260809 02:15:24.690747 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.685601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:48377 (local address 127.9.40.126:33937)
0809 02:15:24.685646 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.685650 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.685903 (+ 253us) server_negotiation.cc:407] Connection header received
0809 02:15:24.686481 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.686482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.686496 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.686548 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.687292 (+ 744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.687810 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.689127 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.689414 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.689490 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.690449 (+ 959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:24.690451 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:24.690451 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:24.690456 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:24.690471 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:24.690476 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:24.690526 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:24.690636 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 02:15:24.690664 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:15:24.690987 19954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.685579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33937 (local address 127.9.40.66:48377)
0809 02:15:24.685774 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:24.685809 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:24.686358 (+ 549us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:24.686624 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:24.686628 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:24.686638 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:24.687199 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.687204 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.687935 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.687936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.689020 (+ 1084us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.689023 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.689514 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.689515 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.689654 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.690305 (+ 651us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:24.690306 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:24.690335 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:24.690357 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:24.690532 (+ 175us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:24.690538 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:24.690541 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:24.690639 (+ 98us) client_negotiation.cc:240] Negotiation successful
0809 02:15:24.690745 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":81,"threads_started":1}
I20260809 02:15:24.691519 19952 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33937
I20260809 02:15:24.691627 19952 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:24.691807 19952 heartbeater.cc:511] Master 127.9.40.126:33937 requested a full tablet report, sending...
I20260809 02:15:24.692242 19609 ts_manager.cc:194] Registered new tserver with Master: 9a802c55eeab4f2faefd5ddaf369c564 (127.9.40.66:37139)
I20260809 02:15:24.692631 19609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:48377
W20260809 02:15:24.774814 19955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:24.775019 19955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:24.775043 19955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:24.776242 19955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:24.776294 19955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:24.776319 19955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:24.777737 19955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:33937
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:24.777987 19955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:24.778291 19955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:24.783360 19963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.783337 19966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.783344 19962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.783699 19955 server_base.cc:1034] running on GCE node
I20260809 02:15:24.783906 19955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:24.784160 19955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:24.785326 19955 hybrid_clock.cc:648] HybridClock initialized: now 1786241724785277 us; error 76 us; skew 500 ppm
I20260809 02:15:24.786689 19955 webserver.cc:460] Webserver started at http://127.9.40.67:33839/ using document root <none> and password file <none>
I20260809 02:15:24.786959 19955 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:24.787056 19955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:24.787156 19955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:24.788442 19955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "6a6f63284d8c417390da681ee591e5f2"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "9450ee63f4d4bddbf23a00bdd0d0225b"
server_key_iv: "b9786d318f2ad7909cb90f2a13b5da85"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.788863 19955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "6a6f63284d8c417390da681ee591e5f2"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "9450ee63f4d4bddbf23a00bdd0d0225b"
server_key_iv: "b9786d318f2ad7909cb90f2a13b5da85"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.790225 19955 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:24.791107 19973 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.791396 19955 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:24.791472 19955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "6a6f63284d8c417390da681ee591e5f2"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "9450ee63f4d4bddbf23a00bdd0d0225b"
server_key_iv: "b9786d318f2ad7909cb90f2a13b5da85"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.791571 19955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:24.805054 19955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:24.805385 19955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:24.805546 19955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:24.805801 19955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:24.806154 19955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:24.806210 19955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.806254 19955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:24.806282 19955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:24.812357 19955 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:45577
I20260809 02:15:24.812402 20094 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:45577 every 8 connection(s)
I20260809 02:15:24.812786 19955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:24.815400 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 19955
I20260809 02:15:24.815481 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:24.815678 9377 external_mini_cluster.cc:1428] Setting key be7ac449defe97f1d8102a97fafa0871
I20260809 02:15:24.818724 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.813549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:44391 (local address 127.9.40.126:33937)
0809 02:15:24.813611 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.813631 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.813841 (+ 210us) server_negotiation.cc:407] Connection header received
0809 02:15:24.814396 (+ 555us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.814397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.814410 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.814481 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.815237 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.815883 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.817314 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.817500 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.817571 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.818484 (+ 913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:24.818485 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:24.818485 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:24.818489 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:24.818501 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:24.818505 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:24.818552 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:24.818641 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:15:24.818663 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:15:24.818924 20097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:24.813506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:33937 (local address 127.9.40.67:44391)
0809 02:15:24.813710 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:24.813753 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:24.814255 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:24.814568 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:24.814571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:24.814580 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:24.815111 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.815116 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.816138 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.816139 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.817209 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:24.817212 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:24.817608 (+ 396us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:24.817609 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:24.817730 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:24.818348 (+ 618us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:24.818348 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:24.818377 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:24.818400 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:24.818563 (+ 163us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:24.818568 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:24.818570 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:15:24.818626 (+ 56us) client_negotiation.cc:240] Negotiation successful
0809 02:15:24.818710 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":103,"threads_started":1}
I20260809 02:15:24.819456 20095 heartbeater.cc:347] Connected to a master server at 127.9.40.126:33937
I20260809 02:15:24.819578 20095 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:24.819796 20095 heartbeater.cc:511] Master 127.9.40.126:33937 requested a full tablet report, sending...
I20260809 02:15:24.820241 19609 ts_manager.cc:194] Registered new tserver with Master: 6a6f63284d8c417390da681ee591e5f2 (127.9.40.67:45577)
I20260809 02:15:24.820616 19609 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:44391
I20260809 02:15:24.827739 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:15:24.829979 19651 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:24.829354 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58336 (local address 127.9.40.126:33937)
0809 02:15:24.829398 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:24.829400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:24.829604 (+ 204us) server_negotiation.cc:407] Connection header received
0809 02:15:24.829675 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:24.829676 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:24.829687 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:24.829758 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:24.829902 (+ 144us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58336: BlockingRecv error: recv got EOF from 127.0.0.1:58336 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:15:24.830143 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19673
I20260809 02:15:24.841914 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19816
I20260809 02:15:24.847947 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19955
I20260809 02:15:24.853823 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 19577
2026-08-09T02:15:24Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (626 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-09T02:15:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:24Z Disabled control of system clock
I20260809 02:15:24.883565 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37995
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:46701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37995
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:15:24.976485 20108 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:24.976822 20108 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:24.976881 20108 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:24.978682 20108 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:24.978747 20108 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:24.978765 20108 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:24.978785 20108 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:24.978806 20108 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:24.981091 20108 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37995
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20108
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:24.981396 20108 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:24.981721 20108 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:24.988848 20116 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.988837 20113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:24.988837 20118 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:24.994004 20108 server_base.cc:1034] running on GCE node
I20260809 02:15:24.994335 20108 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:24.994732 20108 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:24.995934 20108 hybrid_clock.cc:648] HybridClock initialized: now 1786241724995912 us; error 33 us; skew 500 ppm
I20260809 02:15:24.997442 20108 webserver.cc:460] Webserver started at http://127.9.40.126:44603/ using document root <none> and password file <none>
I20260809 02:15:24.997660 20108 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:24.997702 20108 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:24.997795 20108 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:24.998847 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "20a1c73dbdf32bc326ce4b6c07ef7151"
server_key_iv: "33eed8220d55e94a94f4c3900becda29"
server_key_version: "encryptionkey@0"
I20260809 02:15:24.999166 20108 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "20a1c73dbdf32bc326ce4b6c07ef7151"
server_key_iv: "33eed8220d55e94a94f4c3900becda29"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.000622 20108 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:15:25.001667 20124 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.001885 20108 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:25.001986 20108 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c"
format_stamp: "Formatted at 2026-08-09 02:15:24 on dist-test-slave-qg60"
server_key: "20a1c73dbdf32bc326ce4b6c07ef7151"
server_key_iv: "33eed8220d55e94a94f4c3900becda29"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.002059 20108 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:25.005615 20108 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:25.005864 20108 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:25.005968 20108 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:25.010007 20108 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37995
I20260809 02:15:25.010108 20180 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37995 every 8 connection(s)
I20260809 02:15:25.010448 20108 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:25.011322 20181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:25.014654 20181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c: Bootstrap starting.
I20260809 02:15:25.015343 20181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:25.015686 20181 log.cc:824] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:25.016588 20181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c: No bootstrap required, opened a new log
I20260809 02:15:25.018455 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20108
I20260809 02:15:25.018554 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:15:25.018755 9377 external_mini_cluster.cc:1428] Setting key 0a8bed1797d901e90ce461462dc55b7b
I20260809 02:15:25.018680 20181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } }
I20260809 02:15:25.018872 20181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:25.018920 20181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59cb37ecd0344cfc8b5f743bfdbd0f8c, State: Initialized, Role: FOLLOWER
I20260809 02:15:25.019062 20181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } }
I20260809 02:15:25.019119 20181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:25.019140 20181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:25.019166 20181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:25.019964 20181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } }
I20260809 02:15:25.020080 20181 leader_election.cc:302] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59cb37ecd0344cfc8b5f743bfdbd0f8c; no voters:
I20260809 02:15:25.020278 20181 leader_election.cc:288] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:25.020473 20188 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:25.020669 20188 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [term 1 LEADER]: Becoming Leader. State: Replica: 59cb37ecd0344cfc8b5f743bfdbd0f8c, State: Running, Role: LEADER
I20260809 02:15:25.020732 20181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:25.020795 20188 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } }
I20260809 02:15:25.021127 20190 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59cb37ecd0344cfc8b5f743bfdbd0f8c. Latest consensus state: current_term: 1 leader_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } } }
I20260809 02:15:25.021212 20190 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:25.021353 20191 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59cb37ecd0344cfc8b5f743bfdbd0f8c" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37995 } } }
I20260809 02:15:25.021433 20191 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:25.021826 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.019422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45888 (local address 127.9.40.126:37995)
0809 02:15:25.019632 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:25.019641 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:25.019677 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:15:25.019805 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:25.019813 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:25.019863 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:25.020035 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:25.021049 (+ 1014us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:25.021052 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:25.021053 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:25.021057 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:25.021107 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:25.021122 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:25.021177 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:25.021540 (+ 363us) server_negotiation.cc:299] Negotiation successful
0809 02:15:25.021603 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":80,"threads_started":1}
I20260809 02:15:25.022063 20198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:25.022625 20198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:25.024258 20198 catalog_manager.cc:1347] Generated new cluster ID: c55c3be096fb40f78d6e49da2f715e49
I20260809 02:15:25.024302 20198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:25.045779 20198 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:25.045974 20198 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:25.052449 20198 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59cb37ecd0344cfc8b5f743bfdbd0f8c: Generated new TSK 0
I20260809 02:15:25.052675 20198 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:15:25.057138 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--builtin_ntp_servers=127.9.40.84:46701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260809 02:15:25.160149 20208 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:25.160368 20208 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:25.160391 20208 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:25.161669 20208 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:25.161715 20208 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:25.161742 20208 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:25.163153 20208 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20208
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:25.163415 20208 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:25.163741 20208 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:25.169697 20214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:25.169837 20216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:25.169721 20213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:25.170035 20208 server_base.cc:1034] running on GCE node
I20260809 02:15:25.170264 20208 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:25.170514 20208 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:25.171698 20208 hybrid_clock.cc:648] HybridClock initialized: now 1786241725171682 us; error 30 us; skew 500 ppm
I20260809 02:15:25.173410 20208 webserver.cc:460] Webserver started at http://127.9.40.65:39983/ using document root <none> and password file <none>
I20260809 02:15:25.173668 20208 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:25.173727 20208 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:25.173849 20208 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:25.175252 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "7b2973d330884d9e817be7f20798e621"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "9d3309e0bf62537b62a9d5623e8ec42c"
server_key_iv: "d3786e93a67094fb3160e4d78cd9868d"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.175691 20208 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "7b2973d330884d9e817be7f20798e621"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "9d3309e0bf62537b62a9d5623e8ec42c"
server_key_iv: "d3786e93a67094fb3160e4d78cd9868d"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.177335 20208 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 02:15:25.178200 20222 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.178496 20208 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:25.178629 20208 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "7b2973d330884d9e817be7f20798e621"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "9d3309e0bf62537b62a9d5623e8ec42c"
server_key_iv: "d3786e93a67094fb3160e4d78cd9868d"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.178717 20208 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:25.182327 20208 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:25.182607 20208 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:25.182751 20208 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:25.182966 20208 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:25.183290 20208 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:25.183346 20208 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.183383 20208 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:25.183409 20208 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.190018 20208 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:45841
I20260809 02:15:25.190052 20349 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:45841 every 8 connection(s)
I20260809 02:15:25.190407 20208 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:25.193725 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20208
I20260809 02:15:25.193831 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:25.194049 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.191213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:48799 (local address 127.9.40.126:37995)
0809 02:15:25.191303 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:25.191306 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:25.191528 (+ 222us) server_negotiation.cc:407] Connection header received
0809 02:15:25.192131 (+ 603us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:25.192132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:25.192140 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:25.192197 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:25.193691 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:25.193693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:25.193693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:25.193699 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:25.193718 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:25.193723 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:25.193799 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:25.193952 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:15:25.193974 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:15:25.194133 9377 external_mini_cluster.cc:1428] Setting key b71923ca954879514883ff4814a4ee06
I20260809 02:15:25.194352 20352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.191165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37995 (local address 127.9.40.65:48799)
0809 02:15:25.191393 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:25.191430 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:25.191961 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:25.192286 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:25.192290 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:25.192299 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:25.193481 (+ 1182us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:25.193482 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:25.193541 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:25.193584 (+ 43us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:25.193820 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:25.193825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:25.193828 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:25.193954 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:15:25.194022 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":111,"threads_started":1}
I20260809 02:15:25.194903 20350 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37995
I20260809 02:15:25.195017 20350 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:25.195214 20350 heartbeater.cc:511] Master 127.9.40.126:37995 requested a full tablet report, sending...
I20260809 02:15:25.195300 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--builtin_ntp_servers=127.9.40.84:46701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 02:15:25.195791 20141 ts_manager.cc:194] Registered new tserver with Master: 7b2973d330884d9e817be7f20798e621 (127.9.40.65:45841)
W20260809 02:15:25.283098 20353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:25.283305 20353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:25.283329 20353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:25.284421 20353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:25.284456 20353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:25.284480 20353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:25.285990 20353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:25.286199 20353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:25.286466 20353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:25.291546 20359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:25.291585 20358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:25.291548 20361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:25.291872 20353 server_base.cc:1034] running on GCE node
I20260809 02:15:25.292044 20353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:25.292243 20353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:25.293385 20353 hybrid_clock.cc:648] HybridClock initialized: now 1786241725293370 us; error 26 us; skew 500 ppm
I20260809 02:15:25.294636 20353 webserver.cc:460] Webserver started at http://127.9.40.66:44297/ using document root <none> and password file <none>
I20260809 02:15:25.294850 20353 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:25.294891 20353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:25.294983 20353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:25.295991 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "c08c5eb700574917bc7659553847c29e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "bf86e15c9c92c8e5de2edb90b983491a"
server_key_iv: "9ca5a3a8f63889cfaf2d5c514bd804cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.296284 20353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "c08c5eb700574917bc7659553847c29e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "bf86e15c9c92c8e5de2edb90b983491a"
server_key_iv: "9ca5a3a8f63889cfaf2d5c514bd804cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.297631 20353 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:25.298326 20367 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.298478 20353 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:15:25.298578 20353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "c08c5eb700574917bc7659553847c29e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "bf86e15c9c92c8e5de2edb90b983491a"
server_key_iv: "9ca5a3a8f63889cfaf2d5c514bd804cb"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.298633 20353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:25.301927 20353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:25.302137 20353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:25.302229 20353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:25.302378 20353 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:25.302613 20353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:25.302644 20353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.302663 20353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:25.302675 20353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.308465 20353 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39723
I20260809 02:15:25.308557 20492 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39723 every 8 connection(s)
I20260809 02:15:25.308924 20353 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:25.310946 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20353
I20260809 02:15:25.311092 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:25.311329 9377 external_mini_cluster.cc:1428] Setting key 95accb76b6b8e2cff404f1ba93a96330
I20260809 02:15:25.312222 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.309665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:34499 (local address 127.9.40.126:37995)
0809 02:15:25.309727 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:25.309729 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:25.309950 (+ 221us) server_negotiation.cc:407] Connection header received
0809 02:15:25.310475 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:25.310476 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:25.310481 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:25.310543 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:25.311936 (+ 1393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:25.311937 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:25.311937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:25.311943 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:25.311963 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:25.311967 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:25.312075 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:25.312158 (+ 83us) server_negotiation.cc:299] Negotiation successful
0809 02:15:25.312170 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260809 02:15:25.312417 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--builtin_ntp_servers=127.9.40.84:46701
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260809 02:15:25.312481 20496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.309648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37995 (local address 127.9.40.66:34499)
0809 02:15:25.309827 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:25.309860 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:25.310327 (+ 467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:25.310614 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:25.310618 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:25.310625 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:25.311792 (+ 1167us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:25.311793 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:25.311821 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:25.311848 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:25.312056 (+ 208us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:25.312062 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:25.312065 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:25.312171 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 02:15:25.312236 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":96,"threads_started":1}
I20260809 02:15:25.312970 20494 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37995
I20260809 02:15:25.313079 20494 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:25.313225 20494 heartbeater.cc:511] Master 127.9.40.126:37995 requested a full tablet report, sending...
I20260809 02:15:25.313583 20141 ts_manager.cc:194] Registered new tserver with Master: c08c5eb700574917bc7659553847c29e (127.9.40.66:39723)
W20260809 02:15:25.397212 20497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:25.397428 20497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:25.397452 20497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:25.398578 20497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:25.398614 20497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:25.398639 20497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:25.400044 20497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46701
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37995
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:25.400246 20497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:25.400493 20497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:25.405939 20503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:25.406013 20497 server_base.cc:1034] running on GCE node
W20260809 02:15:25.406096 20502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:25.406232 20505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:25.406437 20497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:25.406734 20497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:25.407889 20497 hybrid_clock.cc:648] HybridClock initialized: now 1786241725407882 us; error 41 us; skew 500 ppm
I20260809 02:15:25.409452 20497 webserver.cc:460] Webserver started at http://127.9.40.67:38709/ using document root <none> and password file <none>
I20260809 02:15:25.409715 20497 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:25.409786 20497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:25.409946 20497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:25.411015 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "213352c1481540b88162f6dac8e11a6e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "421c3b53a11604770cb8ad1b25614d01"
server_key_iv: "aca114eaba298661267742fe4e60ea7a"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.411381 20497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "213352c1481540b88162f6dac8e11a6e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "421c3b53a11604770cb8ad1b25614d01"
server_key_iv: "aca114eaba298661267742fe4e60ea7a"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.412855 20497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:15:25.413687 20512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.413869 20497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:25.413969 20497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "213352c1481540b88162f6dac8e11a6e"
format_stamp: "Formatted at 2026-08-09 02:15:25 on dist-test-slave-qg60"
server_key: "421c3b53a11604770cb8ad1b25614d01"
server_key_iv: "aca114eaba298661267742fe4e60ea7a"
server_key_version: "encryptionkey@0"
I20260809 02:15:25.414067 20497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:25.417477 20497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:25.417774 20497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:25.417927 20497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:25.418149 20497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:25.418468 20497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:25.418524 20497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.418556 20497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:25.418582 20497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:25.424903 20497 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:39249
I20260809 02:15:25.424965 20635 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:39249 every 8 connection(s)
I20260809 02:15:25.425328 20497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:15:25.427917 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20497
I20260809 02:15:25.428084 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:25.428289 9377 external_mini_cluster.cc:1428] Setting key 683611798b3c2e5d269287310f4b672b
I20260809 02:15:25.428417 20187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.426165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:55035 (local address 127.9.40.126:37995)
0809 02:15:25.426243 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:25.426245 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:25.426459 (+ 214us) server_negotiation.cc:407] Connection header received
0809 02:15:25.426916 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:25.426918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:25.426924 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:25.427017 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:25.428159 (+ 1142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:25.428161 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:25.428161 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:15:25.428166 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:25.428181 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:15:25.428185 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:25.428245 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:25.428329 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:15:25.428344 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:15:25.428643 20638 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:25.426104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37995 (local address 127.9.40.67:55035)
0809 02:15:25.426338 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:25.426378 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:25.426828 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:25.427033 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:25.427036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:25.427045 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:25.427965 (+ 920us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:15:25.427965 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:25.427995 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:15:25.428024 (+ 29us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:25.428258 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:25.428264 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:25.428267 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:25.428341 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:15:25.428402 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":117,"threads_started":1}
I20260809 02:15:25.429092 20636 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37995
I20260809 02:15:25.429412 20636 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:25.429589 20636 heartbeater.cc:511] Master 127.9.40.126:37995 requested a full tablet report, sending...
I20260809 02:15:25.430063 20141 ts_manager.cc:194] Registered new tserver with Master: 213352c1481540b88162f6dac8e11a6e (127.9.40.67:39249)
I20260809 02:15:25.440590 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260809 02:15:25.442534 20187 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:15:25.442077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45894 (local address 127.9.40.126:37995)
0809 02:15:25.442121 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:25.442123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:25.442249 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:15:25.442292 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:25.442293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:25.442297 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:25.442351 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:25.442477 (+ 126us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:45894: BlockingRecv error: recv got EOF from 127.0.0.1:45894 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:15:25.442696 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20208
I20260809 02:15:25.451502 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20353
I20260809 02:15:25.458845 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20497
I20260809 02:15:25.464663 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20108
2026-08-09T02:15:25Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (612 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:25 dist-test-slave-qg60 krb5kdc[20646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:25 dist-test-slave-qg60 krb5kdc[20646](info): set up 2 sockets
Aug 09 02:15:25 dist-test-slave-qg60 krb5kdc[20646](info): commencing operation
krb5kdc: starting...
W20260809 02:15:26.556914 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.069s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:15:26 dist-test-slave-qg60 krb5kdc[20646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:15:26.684075 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38795
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38795
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:26.771039 20660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:26.771296 20660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:26.771335 20660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:26.772511 20660 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:15:26.772583 20660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:26.772609 20660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:26.772641 20660 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:15:26.772665 20660 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:15:26.774226 20660 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38795
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38795
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:26.774519 20660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:26.774765 20660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:26.779881 20666 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:26.779867 20670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:26.779939 20668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:26.780433 20660 server_base.cc:1034] running on GCE node
I20260809 02:15:26.780640 20660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:26.780946 20660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:26.782121 20660 hybrid_clock.cc:648] HybridClock initialized: now 1786241726782088 us; error 47 us; skew 500 ppm
Aug 09 02:15:26 dist-test-slave-qg60 krb5kdc[20646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241726, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:26.784791 20660 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:15:26.785319 20660 webserver.cc:460] Webserver started at http://127.9.40.126:43771/ using document root <none> and password file <none>
I20260809 02:15:26.785554 20660 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:26.785624 20660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:26.785786 20660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:26.786875 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "840957252c12408badc69bdbec2f0a83"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "4b0c8c5bf9d1baf68bf6a8354400d0d1"
server_key_iv: "6ed1d179f8ef940d0be144fdf4b7bb88"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.787269 20660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "840957252c12408badc69bdbec2f0a83"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "4b0c8c5bf9d1baf68bf6a8354400d0d1"
server_key_iv: "6ed1d179f8ef940d0be144fdf4b7bb88"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.788686 20660 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:26.789589 20679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:26.789788 20660 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:26.789862 20660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "840957252c12408badc69bdbec2f0a83"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "4b0c8c5bf9d1baf68bf6a8354400d0d1"
server_key_iv: "6ed1d179f8ef940d0be144fdf4b7bb88"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.789912 20660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:26.812017 20660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:26.814477 20660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:26.814595 20660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:26.818960 20660 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38795
I20260809 02:15:26.819018 20733 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38795 every 8 connection(s)
I20260809 02:15:26.819360 20660 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:15:26.819732 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20660
I20260809 02:15:26.819831 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:15:26.820037 9377 external_mini_cluster.cc:1428] Setting key 6126a671d3fb90dca1dc821f6e2afafb
I20260809 02:15:26.820574 20734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:26.823879 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: Bootstrap starting.
I20260809 02:15:26.824553 20734 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:26.824906 20734 log.cc:824] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:26.825783 20734 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: No bootstrap required, opened a new log
Aug 09 02:15:26 dist-test-slave-qg60 krb5kdc[20646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:26.828205 20734 raft_consensus.cc:348] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } }
I20260809 02:15:26.828384 20734 raft_consensus.cc:374] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:26.828424 20734 raft_consensus.cc:729] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 840957252c12408badc69bdbec2f0a83, State: Initialized, Role: FOLLOWER
I20260809 02:15:26.828570 20734 consensus_queue.cc:260] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } }
I20260809 02:15:26.828663 20734 raft_consensus.cc:388] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:15:26.828703 20734 raft_consensus.cc:482] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:15:26.828804 20734 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:26.829622 20734 raft_consensus.cc:504] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } }
I20260809 02:15:26.829747 20734 leader_election.cc:302] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 840957252c12408badc69bdbec2f0a83; no voters:
I20260809 02:15:26.829921 20734 leader_election.cc:288] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:15:26.829981 20739 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:26.830143 20739 raft_consensus.cc:686] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [term 1 LEADER]: Becoming Leader. State: Replica: 840957252c12408badc69bdbec2f0a83, State: Running, Role: LEADER
I20260809 02:15:26.830193 20734 sys_catalog.cc:564] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:15:26.830406 20739 consensus_queue.cc:237] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } }
I20260809 02:15:26.830785 20741 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 840957252c12408badc69bdbec2f0a83. Latest consensus state: current_term: 1 leader_uuid: "840957252c12408badc69bdbec2f0a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } } }
I20260809 02:15:26.830807 20740 sys_catalog.cc:455] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "840957252c12408badc69bdbec2f0a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "840957252c12408badc69bdbec2f0a83" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38795 } } }
I20260809 02:15:26.830991 20740 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:26.831218 20741 sys_catalog.cc:458] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83 [sys.catalog]: This master's current role is: LEADER
I20260809 02:15:26.831406 20748 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:15:26.831990 20748 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:15:26.832408 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:26.820622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57486 (local address 127.9.40.126:38795)
0809 02:15:26.820838 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:26.820847 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:26.820880 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:15:26.821400 (+ 520us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:26.821408 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:26.821472 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:26.821624 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:26.822202 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:26.823579 (+ 1377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:26.824367 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:26.825012 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:26.825150 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:26.827826 (+ 2676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:26.827832 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:26.827832 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:26.827847 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:26.830111 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:26.830595 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:26.830598 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:26.830599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:26.830726 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:26.830950 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:26.830951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:26.830951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:26.831289 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:26.831451 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:26.832149 (+ 698us) server_negotiation.cc:299] Negotiation successful
0809 02:15:26.832260 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":114,"threads_started":1}
I20260809 02:15:26.834050 20748 catalog_manager.cc:1347] Generated new cluster ID: 7fa2a692dea24f498653e69f73637467
I20260809 02:15:26.834111 20748 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:15:26.843048 20748 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:15:26.843636 20748 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:15:26.848404 20748 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: Generated new TSK 0
I20260809 02:15:26.848603 20748 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:15:26.880055 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:26.968076 20762 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:26.968281 20762 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:26.968304 20762 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:26.969544 20762 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:26.969592 20762 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:26.969619 20762 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:15:26.971092 20762 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20762
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:26.971345 20762 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:26.971587 20762 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:26.977515 20769 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:26.977490 20771 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:26.977494 20768 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:26.977851 20762 server_base.cc:1034] running on GCE node
I20260809 02:15:26.978127 20762 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:26.978372 20762 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:26.979540 20762 hybrid_clock.cc:648] HybridClock initialized: now 1786241726979531 us; error 36 us; skew 500 ppm
Aug 09 02:15:26 dist-test-slave-qg60 krb5kdc[20646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241726, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:26.982342 20762 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:15:26.982805 20762 webserver.cc:460] Webserver started at http://127.9.40.65:41361/ using document root <none> and password file <none>
I20260809 02:15:26.983027 20762 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:26.983074 20762 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:26.983240 20762 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:26.984321 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "25e6c559e9344023aae1bb339ee28fcd"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "3457690519de9582be9065b72a6cf621"
server_key_iv: "fd76461fbeca800f6acbd1cd3a8fd4e1"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.984709 20762 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "25e6c559e9344023aae1bb339ee28fcd"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "3457690519de9582be9065b72a6cf621"
server_key_iv: "fd76461fbeca800f6acbd1cd3a8fd4e1"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.986142 20762 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:26.986948 20778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:26.987152 20762 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:26.987257 20762 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "25e6c559e9344023aae1bb339ee28fcd"
format_stamp: "Formatted at 2026-08-09 02:15:26 on dist-test-slave-qg60"
server_key: "3457690519de9582be9065b72a6cf621"
server_key_iv: "fd76461fbeca800f6acbd1cd3a8fd4e1"
server_key_version: "encryptionkey@0"
I20260809 02:15:26.987331 20762 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:26.995129 20762 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:26.997244 20762 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:26.997409 20762 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:26.997655 20762 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:26.997990 20762 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:26.998045 20762 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:26.998080 20762 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:26.998111 20762 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.004184 20762 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:32839
I20260809 02:15:27.004310 20898 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:32839 every 8 connection(s)
I20260809 02:15:27.004608 20762 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:15:27.006340 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20762
I20260809 02:15:27.006446 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:15:27.006651 9377 external_mini_cluster.cc:1428] Setting key 1e7d432f33f4bfa894ba4f9d0046dc0b
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[20646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241726, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:15:27.015906 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.005483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:54933 (local address 127.9.40.126:38795)
0809 02:15:27.005571 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.005573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.005824 (+ 251us) server_negotiation.cc:407] Connection header received
0809 02:15:27.006366 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.006367 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.006381 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.006464 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:27.007846 (+ 1382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.008345 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.009785 (+ 1440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.009993 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.010089 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.012823 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:27.012824 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:27.012824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:27.012831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:27.014289 (+ 1458us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.014782 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.014782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.014783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.014834 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.015132 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.015132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.015132 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.015238 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:27.015289 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.015663 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.015811 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:15:27.016155 20901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.005441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38795 (local address 127.9.40.65:54933)
0809 02:15:27.005679 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.005727 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.006261 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.006643 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.006647 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.007413 (+ 766us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:27.007716 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.007723 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.008448 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.008450 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.009689 (+ 1239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.009692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.010100 (+ 408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.010101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.010223 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.010775 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:27.010783 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:27.012651 (+ 1868us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:27.014380 (+ 1729us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.014388 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.014390 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.014704 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.014903 (+ 199us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.014904 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.014904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.015059 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.015307 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:27.015313 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:27.015486 (+ 173us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.015607 (+ 121us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.015798 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":152,"threads_started":1}
I20260809 02:15:27.016705 20899 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38795
I20260809 02:15:27.016850 20899 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:27.017091 20899 heartbeater.cc:511] Master 127.9.40.126:38795 requested a full tablet report, sending...
I20260809 02:15:27.017647 20697 ts_manager.cc:194] Registered new tserver with Master: 25e6c559e9344023aae1bb339ee28fcd (127.9.40.65:32839)
I20260809 02:15:27.018410 20697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:54933
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:15:27.035890 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:27.122094 20904 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:27.122305 20904 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:27.122331 20904 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:27.123435 20904 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:27.123481 20904 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:27.123507 20904 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:15:27.124956 20904 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20904
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:27.125149 20904 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:27.125336 20904 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:27.130529 20912 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:27.130546 20909 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:27.130537 20910 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:27.130790 20904 server_base.cc:1034] running on GCE node
I20260809 02:15:27.131027 20904 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:27.131279 20904 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:27.132503 20904 hybrid_clock.cc:648] HybridClock initialized: now 1786241727132482 us; error 35 us; skew 500 ppm
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[20646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:27.135304 20904 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:15:27.135756 20904 webserver.cc:460] Webserver started at http://127.9.40.66:36423/ using document root <none> and password file <none>
I20260809 02:15:27.136020 20904 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:27.136092 20904 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:27.136247 20904 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:27.137403 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "40721d5b25664b3793ef234ed993220d"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "73cfd30995942ad7edbab910728df920"
server_key_iv: "da400dd0ebb3fb07741036245f4b6510"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.137773 20904 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "40721d5b25664b3793ef234ed993220d"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "73cfd30995942ad7edbab910728df920"
server_key_iv: "da400dd0ebb3fb07741036245f4b6510"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.139197 20904 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:15:27.139997 20919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.140197 20904 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:27.140308 20904 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "40721d5b25664b3793ef234ed993220d"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "73cfd30995942ad7edbab910728df920"
server_key_iv: "da400dd0ebb3fb07741036245f4b6510"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.140394 20904 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:27.186405 20904 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:27.188987 20904 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:27.189153 20904 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:27.189406 20904 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:27.189733 20904 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:27.189788 20904 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.189834 20904 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:27.189859 20904 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.196286 20904 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:44469
I20260809 02:15:27.196369 21042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:44469 every 8 connection(s)
I20260809 02:15:27.196707 20904 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:15:27.202299 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 20904
I20260809 02:15:27.202435 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:15:27.202716 9377 external_mini_cluster.cc:1428] Setting key 59e5f923bfbe00fdc790933a58a7d30a
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[20646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:27.207686 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.197549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:34027 (local address 127.9.40.126:38795)
0809 02:15:27.197620 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.197623 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.197861 (+ 238us) server_negotiation.cc:407] Connection header received
0809 02:15:27.198432 (+ 571us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.198433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.198443 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.198501 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:27.199460 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.199959 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.201093 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.201319 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.201393 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.203948 (+ 2555us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:27.203950 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:27.203950 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:27.203959 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:27.205895 (+ 1936us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.206456 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.206457 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.206457 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.206532 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.206909 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.206910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.206910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.207095 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:27.207170 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.207527 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.207613 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:15:27.208014 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.197522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38795 (local address 127.9.40.66:34027)
0809 02:15:27.197732 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.197772 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.198333 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.198564 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.198568 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.199177 (+ 609us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:27.199379 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.199384 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.200133 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.200134 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.200985 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.200988 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.201408 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.201409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.201522 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.202071 (+ 549us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:27.202082 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:27.203843 (+ 1761us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:27.206000 (+ 2157us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.206007 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.206010 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.206361 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.206610 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.206611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.206611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.206764 (+ 153us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.207191 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:27.207199 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:27.207375 (+ 176us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.207497 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.207685 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":107,"threads_started":1}
I20260809 02:15:27.208721 21043 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38795
I20260809 02:15:27.208855 21043 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:27.209074 21043 heartbeater.cc:511] Master 127.9.40.126:38795 requested a full tablet report, sending...
I20260809 02:15:27.209558 20697 ts_manager.cc:194] Registered new tserver with Master: 40721d5b25664b3793ef234ed993220d (127.9.40.66:44469)
I20260809 02:15:27.209991 20697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:34027
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:15:27.231948 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:46057
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:15:27.315300 21048 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:15:27.315634 21048 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:15:27.315680 21048 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:15:27.316833 21048 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:15:27.316895 21048 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:15:27.316941 21048 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:15:27.318403 21048 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:46057
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38795
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21048
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:15:27.318686 21048 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:15:27.318924 21048 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:15:27.324007 21056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:15:27.324013 21053 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:27.324156 21048 server_base.cc:1034] running on GCE node
W20260809 02:15:27.324118 21054 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:15:27.324503 21048 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:15:27.324720 21048 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:15:27.326030 21048 hybrid_clock.cc:648] HybridClock initialized: now 1786241727326021 us; error 37 us; skew 500 ppm
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[20646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:15:27.328572 21048 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:15:27.329102 21048 webserver.cc:460] Webserver started at http://127.9.40.67:45361/ using document root <none> and password file <none>
I20260809 02:15:27.329406 21048 fs_manager.cc:359] Metadata directory not provided
I20260809 02:15:27.329473 21048 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:15:27.329629 21048 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:15:27.331058 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "d8276832815c4e50adefd98f64f6d2ce"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "facff6a68ec2de335a678d453ac8fd7b"
server_key_iv: "46518f469af5bbb6f57605289e73e67c"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.331367 21048 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "d8276832815c4e50adefd98f64f6d2ce"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "facff6a68ec2de335a678d453ac8fd7b"
server_key_iv: "46518f469af5bbb6f57605289e73e67c"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.332887 21048 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:15:27.333698 21064 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.333890 21048 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:15:27.334028 21048 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "d8276832815c4e50adefd98f64f6d2ce"
format_stamp: "Formatted at 2026-08-09 02:15:27 on dist-test-slave-qg60"
server_key: "facff6a68ec2de335a678d453ac8fd7b"
server_key_iv: "46518f469af5bbb6f57605289e73e67c"
server_key_version: "encryptionkey@0"
I20260809 02:15:27.334095 21048 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:15:27.379304 21048 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:15:27.381871 21048 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:15:27.381991 21048 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:15:27.382226 21048 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:15:27.382539 21048 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:15:27.382589 21048 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.382613 21048 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:15:27.382665 21048 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:15:27.389092 21048 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:40469
I20260809 02:15:27.389137 21186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:40469 every 8 connection(s)
I20260809 02:15:27.389460 21048 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[20646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241727, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:15:27.398577 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 21048
I20260809 02:15:27.398690 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:15:27.398921 9377 external_mini_cluster.cc:1428] Setting key d0e5dc8ca4e8f419704da76f10e2d751
I20260809 02:15:27.400630 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.390355 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:60841 (local address 127.9.40.126:38795)
0809 02:15:27.390416 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.390418 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.390690 (+ 272us) server_negotiation.cc:407] Connection header received
0809 02:15:27.391295 (+ 605us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.391297 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.391308 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.391369 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:27.392393 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.393068 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.394226 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.394435 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.394506 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.396966 (+ 2460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:27.396967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:27.396968 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:27.396975 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:27.398795 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.399476 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.399485 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.399486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.399569 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.399877 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.399878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.399878 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.400014 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:27.400099 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.400434 (+ 335us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.400528 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:15:27.400902 21189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.390326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38795 (local address 127.9.40.67:60841)
0809 02:15:27.390519 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.390561 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.391189 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.391441 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.391444 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.392074 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:15:27.392272 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.392277 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.393179 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.393180 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.394121 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.394124 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.394540 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.394542 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.394654 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.395190 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:15:27.395198 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:15:27.396876 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:15:27.398888 (+ 2012us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.398895 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.398897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.399390 (+ 493us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.399655 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:15:27.399656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:15:27.399656 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:15:27.399782 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:15:27.400102 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:15:27.400110 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:15:27.400264 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.400395 (+ 131us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.400582 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260809 02:15:27.401504 21187 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38795
I20260809 02:15:27.401611 21187 heartbeater.cc:464] Registering TS with master...
I20260809 02:15:27.401845 21187 heartbeater.cc:511] Master 127.9.40.126:38795 requested a full tablet report, sending...
I20260809 02:15:27.402302 20697 ts_manager.cc:194] Registered new tserver with Master: d8276832815c4e50adefd98f64f6d2ce (127.9.40.67:40469)
I20260809 02:15:27.402726 20697 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:60841
I20260809 02:15:27.410897 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:15:27.418871 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.412431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57490 (local address 127.9.40.126:38795)
0809 02:15:27.412498 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.412503 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.412619 (+ 116us) server_negotiation.cc:407] Connection header received
0809 02:15:27.412661 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.412662 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.412671 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.412780 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:15:27.413371 (+ 591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.413914 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.414655 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.414857 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.414919 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.416132 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:15:27.416134 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:15:27.416134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:15:27.416141 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:15:27.417804 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.418148 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.418149 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.418149 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.418209 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:15:27.418375 (+ 166us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:15:27.418376 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:15:27.418376 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:15:27.418551 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:15:27.418632 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.418731 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.418802 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:15:27.420696 20697 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57490:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:15:27.422385 20697 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:15:27.429075 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.424911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41856 (local address 127.9.40.67:40469)
0809 02:15:27.425181 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.425187 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.425198 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:27.425329 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.425332 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.425349 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.425500 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:27.425663 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.426771 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428056 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.428602 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428661 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.428700 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.428930 (+ 230us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.428976 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":233,"thread_start_us":150,"threads_started":1}
I20260809 02:15:27.429064 21201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.424721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40469 (local address 127.0.0.1:41856)
0809 02:15:27.425118 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.425131 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.425200 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.425465 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.425467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.425469 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:27.425556 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.425558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.426883 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.426883 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.427963 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.427965 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.428697 (+ 732us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.428809 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.428824 (+ 15us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.428898 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.428976 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":370,"mutex_wait_us":17,"thread_start_us":99,"threads_started":1}
I20260809 02:15:27.429162 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.424886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59600 (local address 127.9.40.66:44469)
0809 02:15:27.425647 (+ 761us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.425652 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.425662 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:15:27.425701 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.425703 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.425718 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.425792 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:27.426042 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.427050 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428167 (+ 1117us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.428677 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428778 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.428838 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.429031 (+ 193us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.429076 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":721,"thread_start_us":84,"threads_started":1}
I20260809 02:15:27.429308 21198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.424721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:44469 (local address 127.0.0.1:59600)
0809 02:15:27.425087 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.425131 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.425199 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.425878 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.425879 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.425881 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:27.425929 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.425931 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.427160 (+ 1229us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.427161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.428082 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.428084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.428796 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428797 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.428861 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.428865 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.428931 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.428978 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":58,"threads_started":1}
I20260809 02:15:27.430125 20971 tablet_service.cc:1467] Processing CreateTablet for tablet 9978a03a07ac41d98641a81524734092 (DEFAULT_TABLE table=test-table [id=99a286b6ea7e44ed9740b6df42dd7ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:27.430514 20971 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9978a03a07ac41d98641a81524734092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:27.431366 21116 tablet_service.cc:1467] Processing CreateTablet for tablet 9978a03a07ac41d98641a81524734092 (DEFAULT_TABLE table=test-table [id=99a286b6ea7e44ed9740b6df42dd7ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:27.431690 21116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9978a03a07ac41d98641a81524734092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:27.433089 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.424942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:32839 (local address 127.0.0.1:38446)
0809 02:15:27.425245 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.425254 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.425295 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.425465 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.425467 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.425473 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:27.425556 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.425558 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.426647 (+ 1089us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.426648 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.427954 (+ 1306us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.427957 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.432777 (+ 4820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.432778 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.432860 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.432864 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.432936 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.432986 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"mutex_wait_us":5,"thread_start_us":80,"threads_started":1}
I20260809 02:15:27.433085 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.425072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.9.40.65:32839)
0809 02:15:27.425287 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.425292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.425303 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:27.425346 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.425348 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.425362 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.425444 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:27.425772 (+ 328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.426537 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.428782 (+ 2245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.429550 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.429643 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.429720 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.432956 (+ 3236us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.433002 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
I20260809 02:15:27.433934 20829 tablet_service.cc:1467] Processing CreateTablet for tablet 9978a03a07ac41d98641a81524734092 (DEFAULT_TABLE table=test-table [id=99a286b6ea7e44ed9740b6df42dd7ce2]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:15:27.434074 21204 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Bootstrap starting.
I20260809 02:15:27.434276 20829 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9978a03a07ac41d98641a81524734092. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:15:27.434902 21204 tablet_bootstrap.cc:654] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:27.434919 21205 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Bootstrap starting.
I20260809 02:15:27.435577 21205 tablet_bootstrap.cc:654] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:27.435562 21204 log.cc:824] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:27.436214 21205 log.cc:824] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:27.436506 21204 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: No bootstrap required, opened a new log
I20260809 02:15:27.436587 21204 ts_tablet_manager.cc:1397] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:27.436987 21205 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: No bootstrap required, opened a new log
I20260809 02:15:27.437078 21205 ts_tablet_manager.cc:1397] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:15:27.437776 21208 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Bootstrap starting.
I20260809 02:15:27.438715 21208 tablet_bootstrap.cc:654] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Neither blocks nor log segments found. Creating new log.
I20260809 02:15:27.438704 21204 raft_consensus.cc:348] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.438875 21204 raft_consensus.cc:374] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:27.438920 21204 raft_consensus.cc:729] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40721d5b25664b3793ef234ed993220d, State: Initialized, Role: FOLLOWER
I20260809 02:15:27.439077 21204 consensus_queue.cc:260] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.439144 21205 raft_consensus.cc:348] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.439265 21208 log.cc:824] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:15:27.439311 21205 raft_consensus.cc:374] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:27.439360 21205 raft_consensus.cc:729] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d8276832815c4e50adefd98f64f6d2ce, State: Initialized, Role: FOLLOWER
I20260809 02:15:27.439366 21204 ts_tablet_manager.cc:1428] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:15:27.439481 21043 heartbeater.cc:503] Master 127.9.40.126:38795 was elected leader, sending a full tablet report...
I20260809 02:15:27.439512 21205 consensus_queue.cc:260] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.439805 21205 ts_tablet_manager.cc:1428] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260809 02:15:27.439826 21187 heartbeater.cc:503] Master 127.9.40.126:38795 was elected leader, sending a full tablet report...
I20260809 02:15:27.440693 21208 tablet_bootstrap.cc:492] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: No bootstrap required, opened a new log
I20260809 02:15:27.440793 21208 ts_tablet_manager.cc:1397] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:15:27.442145 21208 raft_consensus.cc:348] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.442265 21208 raft_consensus.cc:374] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:15:27.442289 21208 raft_consensus.cc:729] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 25e6c559e9344023aae1bb339ee28fcd, State: Initialized, Role: FOLLOWER
I20260809 02:15:27.442418 21208 consensus_queue.cc:260] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.442693 21208 ts_tablet_manager.cc:1428] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:15:27.442759 20899 heartbeater.cc:503] Master 127.9.40.126:38795 was elected leader, sending a full tablet report...
W20260809 02:15:27.447350 21044 tablet.cc:2369] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:15:27.505426 20900 tablet.cc:2369] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:15:27.579797 21209 raft_consensus.cc:482] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:15:27.579922 21209 raft_consensus.cc:504] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.580289 21209 leader_election.cc:288] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d8276832815c4e50adefd98f64f6d2ce (127.9.40.67:40469), 25e6c559e9344023aae1bb339ee28fcd (127.9.40.65:32839)
I20260809 02:15:27.584416 21203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.580636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:44639 (local address 127.9.40.65:32839)
0809 02:15:27.580681 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.580684 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.580800 (+ 116us) server_negotiation.cc:407] Connection header received
0809 02:15:27.580833 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.580834 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.580847 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.580910 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:27.581084 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.581970 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.583597 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.584004 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.584101 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.584112 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.584281 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.584330 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:15:27.584427 21202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.580610 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:57157 (local address 127.9.40.67:40469)
0809 02:15:27.580664 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.580666 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.580677 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:27.580709 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.580710 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.580770 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.580854 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:15:27.581084 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.581937 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.583628 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.583998 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.584060 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.584097 (+ 37us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.584310 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.584355 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 02:15:27.584439 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.580495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:40469 (local address 127.9.40.66:57157)
0809 02:15:27.580555 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.580568 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.580615 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.580898 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.580899 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.580901 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:27.580997 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.580999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.582111 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.582112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.583474 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.583478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.584091 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.584092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.584207 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.584210 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.584294 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.584338 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40,"mutex_wait_us":68}
I20260809 02:15:27.584446 21215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.580495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:32839 (local address 127.9.40.66:44639)
0809 02:15:27.580713 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.580721 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:15:27.580782 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:15:27.580923 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:15:27.580924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:15:27.580925 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:15:27.580997 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.580999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.582111 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.582112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.583474 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:15:27.583478 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.584127 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.584128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:15:27.584195 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.584200 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:15:27.584245 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 02:15:27.584310 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":103,"thread_start_us":128,"threads_started":1}
I20260809 02:15:27.584873 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9978a03a07ac41d98641a81524734092" candidate_uuid: "40721d5b25664b3793ef234ed993220d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8276832815c4e50adefd98f64f6d2ce" is_pre_election: true
I20260809 02:15:27.585094 21137 raft_consensus.cc:2454] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40721d5b25664b3793ef234ed993220d in term 0.
I20260809 02:15:27.585116 20850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9978a03a07ac41d98641a81524734092" candidate_uuid: "40721d5b25664b3793ef234ed993220d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e6c559e9344023aae1bb339ee28fcd" is_pre_election: true
I20260809 02:15:27.585256 20850 raft_consensus.cc:2454] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40721d5b25664b3793ef234ed993220d in term 0.
I20260809 02:15:27.585367 20922 leader_election.cc:302] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40721d5b25664b3793ef234ed993220d, d8276832815c4e50adefd98f64f6d2ce; no voters:
I20260809 02:15:27.585512 21209 raft_consensus.cc:2781] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:15:27.585598 21209 raft_consensus.cc:482] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:15:27.585637 21209 raft_consensus.cc:3037] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:27.586436 21209 raft_consensus.cc:504] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.586609 21209 leader_election.cc:288] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [CANDIDATE]: Term 1 election: Requested vote from peers d8276832815c4e50adefd98f64f6d2ce (127.9.40.67:40469), 25e6c559e9344023aae1bb339ee28fcd (127.9.40.65:32839)
I20260809 02:15:27.586767 21137 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9978a03a07ac41d98641a81524734092" candidate_uuid: "40721d5b25664b3793ef234ed993220d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d8276832815c4e50adefd98f64f6d2ce"
I20260809 02:15:27.586799 20850 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9978a03a07ac41d98641a81524734092" candidate_uuid: "40721d5b25664b3793ef234ed993220d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "25e6c559e9344023aae1bb339ee28fcd"
I20260809 02:15:27.586867 21137 raft_consensus.cc:3037] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:27.586879 20850 raft_consensus.cc:3037] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:15:27.587684 20850 raft_consensus.cc:2454] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40721d5b25664b3793ef234ed993220d in term 1.
I20260809 02:15:27.587687 21137 raft_consensus.cc:2454] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40721d5b25664b3793ef234ed993220d in term 1.
I20260809 02:15:27.587893 20922 leader_election.cc:302] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40721d5b25664b3793ef234ed993220d, d8276832815c4e50adefd98f64f6d2ce; no voters:
I20260809 02:15:27.587998 21209 raft_consensus.cc:2781] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:15:27.588153 21209 raft_consensus.cc:686] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 1 LEADER]: Becoming Leader. State: Replica: 40721d5b25664b3793ef234ed993220d, State: Running, Role: LEADER
I20260809 02:15:27.588374 21209 consensus_queue.cc:237] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } }
I20260809 02:15:27.589331 20697 catalog_manager.cc:5495] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d reported cstate change: term changed from 0 to 1, leader changed from <none> to 40721d5b25664b3793ef234ed993220d (127.9.40.66). New cstate: current_term: 1 leader_uuid: "40721d5b25664b3793ef234ed993220d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40721d5b25664b3793ef234ed993220d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 44469 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 } health_report { overall_health: UNKNOWN } } }
I20260809 02:15:27.613994 21200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.611176 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59604 (local address 127.9.40.66:44469)
0809 02:15:27.611251 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:15:27.611253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:15:27.611264 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:15:27.611368 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:15:27.611369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:15:27.611438 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:15:27.611523 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:15:27.611673 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.612199 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.613148 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:15:27.613351 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:15:27.613445 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:15:27.613616 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:15:27.613730 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:15:27.613832 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:15:27.613852 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 02:15:27.613874 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:15:27.615869 21137 raft_consensus.cc:1261] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 1 FOLLOWER]: Refusing update from remote peer 40721d5b25664b3793ef234ed993220d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:27.615872 20850 raft_consensus.cc:1261] T 9978a03a07ac41d98641a81524734092 P 25e6c559e9344023aae1bb339ee28fcd [term 1 FOLLOWER]: Refusing update from remote peer 40721d5b25664b3793ef234ed993220d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:15:27.616276 21216 consensus_queue.cc:1035] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [LEADER]: Connected to new peer: Peer: permanent_uuid: "d8276832815c4e50adefd98f64f6d2ce" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 40469 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:27.617211 21216 consensus_queue.cc:1035] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [LEADER]: Connected to new peer: Peer: permanent_uuid: "25e6c559e9344023aae1bb339ee28fcd" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 32839 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:15:27.619740 21219 mvcc.cc:205] Tried to move back new op lower bound from 7316446116311760896 to 7316446116202799104. Current Snapshot: MvccSnapshot[applied={T|T < 7316446116311760896}]
I20260809 02:15:27.619892 21222 mvcc.cc:205] Tried to move back new op lower bound from 7316446116311760896 to 7316446116202799104. Current Snapshot: MvccSnapshot[applied={T|T < 7316446116311760896}]
I20260809 02:15:27.620414 21220 mvcc.cc:205] Tried to move back new op lower bound from 7316446116311760896 to 7316446116202799104. Current Snapshot: MvccSnapshot[applied={T|T < 7316446116311760896}]
I20260809 02:15:27.622171 20696 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:27.622283 20696 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57490:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:15:27.623147 20696 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 840957252c12408badc69bdbec2f0a83: Sending DeleteTablet for 3 replicas of tablet 9978a03a07ac41d98641a81524734092
I20260809 02:15:27.623639 20829 tablet_service.cc:1514] Processing DeleteTablet for tablet 9978a03a07ac41d98641a81524734092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:27 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:38446
I20260809 02:15:27.623639 21116 tablet_service.cc:1514] Processing DeleteTablet for tablet 9978a03a07ac41d98641a81524734092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:27 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:41856
I20260809 02:15:27.623888 20971 tablet_service.cc:1514] Processing DeleteTablet for tablet 9978a03a07ac41d98641a81524734092 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:15:27 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59600
I20260809 02:15:27.624274 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20762
I20260809 02:15:27.624217 21233 tablet_replica.cc:330] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: stopping tablet replica
I20260809 02:15:27.628304 21233 raft_consensus.cc:2229] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:15:27.628424 21233 raft_consensus.cc:2258] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:27.628804 21231 tablet_replica.cc:330] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: stopping tablet replica
I20260809 02:15:27.628926 21233 ts_tablet_manager.cc:1905] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:15:27.628944 21231 raft_consensus.cc:2229] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:15:27.629088 21231 raft_consensus.cc:2258] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:15:27.631358 21233 ts_tablet_manager.cc:1918] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:27.631453 21233 log.cc:1196] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/9978a03a07ac41d98641a81524734092
I20260809 02:15:27.631796 21233 ts_tablet_manager.cc:1939] T 9978a03a07ac41d98641a81524734092 P 40721d5b25664b3793ef234ed993220d: Deleting consensus metadata
I20260809 02:15:27.632334 20683 catalog_manager.cc:4841] TS 40721d5b25664b3793ef234ed993220d (127.9.40.66:44469): tablet 9978a03a07ac41d98641a81524734092 (table test-table [id=99a286b6ea7e44ed9740b6df42dd7ce2]) successfully deleted
W20260809 02:15:27.633641 20683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:32839 (error 108)
I20260809 02:15:27.634198 21199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:15:27.634075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:32839 (local address 127.0.0.1:38456)
0809 02:15:27.634125 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:15:27.634162 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:32839: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:15:27.634306 21231 ts_tablet_manager.cc:1905] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:15:27.634310 20683 catalog_manager.cc:4584] TS 25e6c559e9344023aae1bb339ee28fcd (127.9.40.65:32839): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9978a03a07ac41d98641a81524734092: Network error: Client connection negotiation failed: client connection to 127.9.40.65:32839: connect: Connection refused (error 111)
I20260809 02:15:27.635694 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20904
I20260809 02:15:27.635732 21231 ts_tablet_manager.cc:1918] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:15:27.635833 21231 log.cc:1196] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/9978a03a07ac41d98641a81524734092
I20260809 02:15:27.636117 21231 ts_tablet_manager.cc:1939] T 9978a03a07ac41d98641a81524734092 P d8276832815c4e50adefd98f64f6d2ce: Deleting consensus metadata
I20260809 02:15:27.638126 20682 catalog_manager.cc:4841] TS d8276832815c4e50adefd98f64f6d2ce (127.9.40.67:40469): tablet 9978a03a07ac41d98641a81524734092 (table test-table [id=99a286b6ea7e44ed9740b6df42dd7ce2]) successfully deleted
I20260809 02:15:27.642282 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 21048
I20260809 02:15:27.650019 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 20660
2026-08-09T02:15:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2184 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[21237](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[21237](info): set up 2 sockets
Aug 09 02:15:27 dist-test-slave-qg60 krb5kdc[21237](info): commencing operation
krb5kdc: starting...
W20260809 02:15:28.753348 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.081s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:15:28 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241728, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:15:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:15:28Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
I20260809 02:15:28.936345 9377 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQln13t/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/postgres -l logfile start
2026-08-09 02:15:31.796 UTC [21270] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:15:31.796 UTC [21270] LOG: listening on IPv4 address "127.9.40.84", port 42219
2026-08-09 02:15:31.801 UTC [21270] LOG: listening on Unix socket "/tmp/.s.PGSQL.42219"
2026-08-09 02:15:31.838 UTC [21273] LOG: database system was shut down at 2026-08-09 02:15:30 UTC
2026-08-09 02:15:31.843 UTC [21270] LOG: database system is ready to accept connections
I20260809 02:15:32.850610 9377 mini_postgres.cc:96] Postgres bound to 42219
2026-08-09 02:15:32.859 UTC [21282] FATAL: database "slave" does not exist
127.9.40.84:42219 - accepting connections
I20260809 02:15:32.860704 9377 mini_ranger.cc:161] Starting Ranger...
I20260809 02:15:32.879719 9377 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:15:33.480634 9377 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:15:33.480731 9377 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin
2026-08-09 02:15:33,815 [I] DB FLAVOR :POSTGRES
2026-08-09 02:15:33,816 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:15:33,816 [I] Checking connection..
2026-08-09 02:15:33,816 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:15:34,087 [I] Checking connection passed.
2026-08-09 02:15:34,088 [I] --------- Verifying version history table ---------
2026-08-09 02:15:34,088 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:15:34,344 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:15:34,345 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:15:34,617 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:15:34,617 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:15:34,617 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:15:34.891 UTC [21351] WARNING: there is no transaction in progress
2026-08-09 02:15:34,899 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:15:34,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:15:35,180 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:15:35,181 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:15:35,181 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:15:35,462 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:15:35,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-qg60.c.gcp-upstream.internal','N') ;"
2026-08-09 02:15:35,979 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:15:36,245 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:15:36,246 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:15:36,520 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:15:36,521 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:15:36,773 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:15:37,041 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:15:37,041 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:15:38.094 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.112 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.130 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.148 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.164 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.184 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.245 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.252 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.268 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.279 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.290 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.305 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.315 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.323 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.331 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:38.339 UTC [21465] WARNING: there is no transaction in progress
2026-08-09 02:15:39,085 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:15:39,085 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:15:39,352 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:15:39,352 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:15:39,627 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:15:39,628 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:15:39,889 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:15:39,890 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:15:40,162 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:15:40,436 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-qg60.c.gcp-upstream.internal';"
2026-08-09 02:15:40,683 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:15:40,684 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:15:40,943 [I] DB_PATCHES have already been applied
I20260809 02:15:40.949538 9377 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQln13t/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:15:40.949636 9377 mini_ranger.cc:193] Using host: 127.9.40.84
I20260809 02:15:40.953472 9377 mini_ranger.cc:227] Ranger admin URL: http://127.9.40.84:60563
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:15:41 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:15:42.141098 9377 mini_ranger.cc:160] Time spent starting Ranger: real 9.280s user 0.000s sys 0.011s
Aug 09, 2026 2:16:02 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:16:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 09, 2026 2:16:02 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:16:02 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:16:03 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:16:04.220885 9377 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:16:04 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:16:04.311954 9377 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:16:04.322585 9377 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:16:05.311658 9377 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:16:05.311753 9377 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms
I20260809 02:16:05.311774 9377 mini_ranger_kms.cc:227] Using postgres at 127.9.40.84:42219
2026-08-09 02:16:05,548 [I] DB FLAVOR :POSTGRES
2026-08-09 02:16:05,548 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:16:05,548 [I] Checking connection
2026-08-09 02:16:05,548 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:16:05,824 [I] connection success
2026-08-09 02:16:05,824 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:16:05,824 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:16:05,824 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:16:06,097 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:16:06,097 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:16:06,097 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:16:06,097 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:42219/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:16:06,373 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:16:06.838019 9377 mini_ranger_kms.cc:310] Created kms service
I20260809 02:16:07.040099 9377 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:16:07.090086 9377 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:16:07 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241764, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:07.485389 9377 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:16:07.485720 9377 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:16:07.485756 9377 mini_ranger_kms.cc:240] Using host: 127.9.40.84
I20260809 02:16:07.489853 9377 mini_ranger_kms.cc:276] Ranger KMS PID: 21660
I20260809 02:16:07.490007 9377 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.40.84:51233
Aug 09, 2026 2:16:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:16:08 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:16:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:16:08 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:16:09 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:16:09 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260809 02:16:09.944551 9377 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.633s user 0.005s sys 0.210s
I20260809 02:16:09.944676 9377 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:16:09.944784 9377 mini_ranger_kms.cc:401] 127.9.40.84:51233/kms/v1/keys
Aug 09 02:16:10 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 09 02:16:10 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 09 02:16:10 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241770, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 09 02:16:13 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:16:13.175853 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41587
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41587
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:51233/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:13.262812 21716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:13.263056 21716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:13.263082 21716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:13.264317 21716 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:16:13.264365 21716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:13.264380 21716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:13.264393 21716 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:16:13.264405 21716 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:16:13.266281 21716 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.40.84:51233/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41587
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:16:13.266554 21716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:13.266821 21716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:13.272838 21722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:13.273146 21721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:13.273255 21724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:13.273279 21716 server_base.cc:1034] running on GCE node
I20260809 02:16:13.273552 21716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:13.273864 21716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:13.275053 21716 hybrid_clock.cc:648] HybridClock initialized: now 1786241773275027 us; error 41 us; skew 500 ppm
Aug 09 02:16:13 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:13.282778 21716 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:16:13.283332 21716 webserver.cc:460] Webserver started at http://127.9.40.126:36347/ using document root <none> and password file <none>
I20260809 02:16:13.283586 21716 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:13.283627 21716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:13.283759 21716 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:16:13 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:13.331739 21716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "8660caf0279c4188a0ae318605fed204"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-qg60"
server_key: "8ec5b10635d2de9d0a6b3cb8ef6e87b9"
server_key_iv: "93cdc45d835d24d18335c57242dda337"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:13.332158 21716 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "8660caf0279c4188a0ae318605fed204"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-qg60"
server_key: "8ec5b10635d2de9d0a6b3cb8ef6e87b9"
server_key_iv: "93cdc45d835d24d18335c57242dda337"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:13.333828 21716 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:16:13.345870 21731 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:13.346168 21716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:16:13.346276 21716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "8660caf0279c4188a0ae318605fed204"
format_stamp: "Formatted at 2026-08-09 02:16:13 on dist-test-slave-qg60"
server_key: "8ec5b10635d2de9d0a6b3cb8ef6e87b9"
server_key_iv: "93cdc45d835d24d18335c57242dda337"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:13.346401 21716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:13.364784 21716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:13.367126 21716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:13.367295 21716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:13.372581 21716 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41587
I20260809 02:16:13.372603 21786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41587 every 8 connection(s)
I20260809 02:16:13.373108 21716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:16:13.373858 21787 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:13.376303 21787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: Bootstrap starting.
I20260809 02:16:13.376978 21787 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:13.377280 21787 log.cc:824] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:13.378014 21787 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: No bootstrap required, opened a new log
I20260809 02:16:13.380059 21787 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } }
I20260809 02:16:13.380232 21787 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:13.380280 21787 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8660caf0279c4188a0ae318605fed204, State: Initialized, Role: FOLLOWER
I20260809 02:16:13.380452 21787 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } }
I20260809 02:16:13.380558 21787 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:16:13.380600 21787 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:16:13.380650 21787 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:13.381225 21787 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } }
I20260809 02:16:13.381348 21787 leader_election.cc:302] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8660caf0279c4188a0ae318605fed204; no voters:
I20260809 02:16:13.381556 21787 leader_election.cc:288] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:16:13.381592 21791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:13.381726 21791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [term 1 LEADER]: Becoming Leader. State: Replica: 8660caf0279c4188a0ae318605fed204, State: Running, Role: LEADER
I20260809 02:16:13.381834 21787 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:16:13.381872 21791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } }
I20260809 02:16:13.382296 21792 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8660caf0279c4188a0ae318605fed204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } } }
I20260809 02:16:13.382309 21793 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8660caf0279c4188a0ae318605fed204. Latest consensus state: current_term: 1 leader_uuid: "8660caf0279c4188a0ae318605fed204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8660caf0279c4188a0ae318605fed204" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41587 } } }
I20260809 02:16:13.382390 21792 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:13.382417 21793 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204 [sys.catalog]: This master's current role is: LEADER
I20260809 02:16:13.382615 21798 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:16:13.383286 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 21716
I20260809 02:16:13.383397 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:16:13.383636 9377 external_mini_cluster.cc:1428] Setting key a4ef9b2c1ff8f4b720411692c544ad93
I20260809 02:16:13.384128 21798 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:16:13.384683 21787 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:16:13.386008 21798 catalog_manager.cc:1347] Generated new cluster ID: 6806c422d54446fcbe94ca229911c407
I20260809 02:16:13.386112 21798 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:16:13 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241773, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:16:13.405704 21798 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:16:13.406172 21810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:13.384097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59308 (local address 127.9.40.126:41587)
0809 02:16:13.391531 (+ 7434us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:13.391573 (+ 42us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:13.391648 (+ 75us) server_negotiation.cc:407] Connection header received
0809 02:16:13.391960 (+ 312us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:13.391969 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:13.392065 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:13.393402 (+ 1337us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:13.394150 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:13.395901 (+ 1751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:13.397077 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:13.397838 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:13.398301 (+ 463us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:13.401169 (+ 2868us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:13.401174 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:13.401175 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:13.401191 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:13.403667 (+ 2476us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:13.404097 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:13.404100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:13.404100 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:13.404225 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:13.404457 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:13.404458 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:13.404458 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:13.404955 (+ 497us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:13.405610 (+ 655us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:13.405696 (+ 86us) server_negotiation.cc:299] Negotiation successful
0809 02:16:13.405853 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7220,"thread_start_us":80,"threads_started":1}
I20260809 02:16:13.406672 21798 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:16:13.424043 21798 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: Generated new TSK 0
I20260809 02:16:13.424360 21798 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:16:14.172279 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:51233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:14.272929 21835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:14.273154 21835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:14.273182 21835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:14.274487 21835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:14.274531 21835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:14.274560 21835 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:16:14.276206 21835 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.40.84:51233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:16:14.276479 21835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.276823 21835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.282397 21841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.282409 21840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.282691 21835 server_base.cc:1034] running on GCE node
W20260809 02:16:14.282409 21844 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.283035 21835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.283325 21835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.284514 21835 hybrid_clock.cc:648] HybridClock initialized: now 1786241774284493 us; error 39 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.287633 21835 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:16:14.288209 21835 webserver.cc:460] Webserver started at http://127.9.40.65:37791/ using document root <none> and password file <none>
I20260809 02:16:14.288446 21835 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.288513 21835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.288657 21835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:14.304409 21835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "bd7ea5fb0b074e64bb9b8c47412f306f"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "f47b1eef712ffef9115e4632bbc3fbea"
server_key_iv: "bf0101654ce63e260371064f2e787449"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.305400 21835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "bd7ea5fb0b074e64bb9b8c47412f306f"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "f47b1eef712ffef9115e4632bbc3fbea"
server_key_iv: "bf0101654ce63e260371064f2e787449"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.307649 21835 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:16:14.324406 21853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.324677 21835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:16:14.324795 21835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "bd7ea5fb0b074e64bb9b8c47412f306f"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "f47b1eef712ffef9115e4632bbc3fbea"
server_key_iv: "bf0101654ce63e260371064f2e787449"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.324887 21835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.342487 21835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.344642 21835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.344842 21835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.345100 21835 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.345511 21835 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.345566 21835 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.345599 21835 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.345621 21835 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.353368 21835 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:43195
I20260809 02:16:14.353785 21980 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:43195 every 8 connection(s)
I20260809 02:16:14.353993 21835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:16:14.362128 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 21835
I20260809 02:16:14.362241 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:16:14.362475 9377 external_mini_cluster.cc:1428] Setting key de5134c55b05d4d33b746c1891e9d1c0
I20260809 02:16:14.365346 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.354882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:56025 (local address 127.9.40.126:41587)
0809 02:16:14.355031 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.355033 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.355937 (+ 904us) server_negotiation.cc:407] Connection header received
0809 02:16:14.355970 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.355971 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.355985 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.356051 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.356672 (+ 621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.357473 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.359029 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.359303 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.359380 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.361705 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.361706 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.361707 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.361714 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.363583 (+ 1869us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.364129 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.364131 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.364131 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.364204 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.364519 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.364519 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.364520 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.364666 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.364765 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.365160 (+ 395us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.365249 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
I20260809 02:16:14.365505 21984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.354840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41587 (local address 127.9.40.65:56025)
0809 02:16:14.355025 (+ 185us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.355076 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.355792 (+ 716us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.356178 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.356182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.356369 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.356561 (+ 192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.356564 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.357600 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.357601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.358919 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.358922 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.359386 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.359387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.359527 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.360130 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.360138 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.361593 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.363679 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.363687 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.363690 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.364041 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.364283 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.364284 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.364284 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.364441 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.364783 (+ 342us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.364789 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.364987 (+ 198us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.365121 (+ 134us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.365288 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":86,"threads_started":1}
I20260809 02:16:14.365999 21981 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41587
I20260809 02:16:14.366132 21981 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.366320 21981 heartbeater.cc:511] Master 127.9.40.126:41587 requested a full tablet report, sending...
I20260809 02:16:14.366868 21751 ts_manager.cc:194] Registered new tserver with Master: bd7ea5fb0b074e64bb9b8c47412f306f (127.9.40.65:43195)
I20260809 02:16:14.367467 21751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:56025
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:16:14.396589 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:51233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:14.508718 21988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:14.509012 21988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:14.509052 21988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:14.510233 21988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:14.510306 21988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:14.510352 21988 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:16:14.511858 21988 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.40.84:51233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:16:14.512131 21988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.512370 21988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.517719 21995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.517907 21996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.517697 21998 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.518625 21988 server_base.cc:1034] running on GCE node
I20260809 02:16:14.518898 21988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.519207 21988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.520372 21988 hybrid_clock.cc:648] HybridClock initialized: now 1786241774520352 us; error 35 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.523206 21988 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:16:14.523769 21988 webserver.cc:460] Webserver started at http://127.9.40.66:45453/ using document root <none> and password file <none>
I20260809 02:16:14.524042 21988 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.524112 21988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.524261 21988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:14.539399 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "1f801ea2e4234f8ab68e083904860b16"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "ff0f0dfc19155be9d2f296b1546ea861"
server_key_iv: "54b8742679ae526801025db75b44a826"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.539858 21988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "1f801ea2e4234f8ab68e083904860b16"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "ff0f0dfc19155be9d2f296b1546ea861"
server_key_iv: "54b8742679ae526801025db75b44a826"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.543512 21988 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.002s
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.566008 22005 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.566309 21988 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260809 02:16:14.566394 21988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "1f801ea2e4234f8ab68e083904860b16"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "ff0f0dfc19155be9d2f296b1546ea861"
server_key_iv: "54b8742679ae526801025db75b44a826"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.566494 21988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.579624 21988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.582122 21988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.582379 21988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.582645 21988 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.583055 21988 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.583295 21988 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.583411 21988 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.583473 21988 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.590860 21988 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:41141
I20260809 02:16:14.591387 21988 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:16:14.594038 22137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:41141 every 8 connection(s)
I20260809 02:16:14.595844 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 21988
I20260809 02:16:14.595937 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:16:14.596138 9377 external_mini_cluster.cc:1428] Setting key d52527d6333f71c3f8d8bc9b7e44824b
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:16:14.604944 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.593751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43183 (local address 127.9.40.126:41587)
0809 02:16:14.593802 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.593804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.594847 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 02:16:14.595540 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.595542 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.595555 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.595617 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.596341 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.597192 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.598747 (+ 1555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.599057 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.599104 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.601769 (+ 2665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.601770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.601770 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.601778 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.603386 (+ 1608us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.603818 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.603819 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.603819 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.603879 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.604161 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.604162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.604163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.604283 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.604371 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.604752 (+ 381us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.604849 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:16:14.605135 22140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.593748 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41587 (local address 127.9.40.66:43183)
0809 02:16:14.594713 (+ 965us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.594756 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.595388 (+ 632us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.595692 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.595697 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.595947 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.596221 (+ 274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.596227 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.597312 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.597313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.598634 (+ 1321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.598637 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.599198 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.599199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.599341 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.599946 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.599956 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.601647 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.603487 (+ 1840us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.603494 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.603497 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.603732 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.603954 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.603954 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.603955 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.604107 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.604435 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.604441 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.604584 (+ 143us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.604697 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.604867 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":923,"thread_start_us":84,"threads_started":1}
I20260809 02:16:14.605661 22138 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41587
I20260809 02:16:14.605760 22138 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.605964 22138 heartbeater.cc:511] Master 127.9.40.126:41587 requested a full tablet report, sending...
I20260809 02:16:14.606429 21751 ts_manager.cc:194] Registered new tserver with Master: 1f801ea2e4234f8ab68e083904860b16 (127.9.40.66:41141)
I20260809 02:16:14.607052 21751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:43183
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:16:14.629714 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:51233/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43073
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:16:14.720995 22144 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:16:14.721222 22144 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:16:14.721261 22144 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:16:14.722626 22144 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:16:14.722702 22144 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:16:14.722759 22144 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:16:14.724471 22144 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43073
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.40.84:51233/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22144
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:16:14.724891 22144 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:16:14.725204 22144 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:16:14.731542 22151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.731649 22150 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:16:14.731705 22153 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:16:14.732002 22144 server_base.cc:1034] running on GCE node
I20260809 02:16:14.732230 22144 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:16:14.732482 22144 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:16:14.733652 22144 hybrid_clock.cc:648] HybridClock initialized: now 1786241774733639 us; error 25 us; skew 500 ppm
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:14.736438 22144 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:16:14.736982 22144 webserver.cc:460] Webserver started at http://127.9.40.67:45479/ using document root <none> and password file <none>
I20260809 02:16:14.737234 22144 fs_manager.cc:359] Metadata directory not provided
I20260809 02:16:14.737319 22144 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:16:14.737465 22144 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:14.766657 22144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "7b2b302e54b749589068dc6dbfc6399c"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "6d42c12458c9c9e174c3af8aae071711"
server_key_iv: "1be630d0d102c35dcb8ed7e7c9abafe5"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.767153 22144 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "7b2b302e54b749589068dc6dbfc6399c"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "6d42c12458c9c9e174c3af8aae071711"
server_key_iv: "1be630d0d102c35dcb8ed7e7c9abafe5"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.769165 22144 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260809 02:16:14.796414 22161 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.796710 22144 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260809 02:16:14.796888 22144 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "7b2b302e54b749589068dc6dbfc6399c"
format_stamp: "Formatted at 2026-08-09 02:16:14 on dist-test-slave-qg60"
server_key: "6d42c12458c9c9e174c3af8aae071711"
server_key_iv: "1be630d0d102c35dcb8ed7e7c9abafe5"
server_key_version: "kuduclusterkey@0"
I20260809 02:16:14.797060 22144 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:16:14.807117 22144 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:16:14.810781 22144 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:16:14.811003 22144 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:16:14.811295 22144 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:16:14.811717 22144 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:16:14.811810 22144 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.811883 22144 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:16:14.811944 22144 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:16:14.819514 22144 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:36753
I20260809 02:16:14.820122 22144 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:16:14.821300 22282 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:36753 every 8 connection(s)
I20260809 02:16:14.826227 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 22144
I20260809 02:16:14.826335 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:16:14.826524 9377 external_mini_cluster.cc:1428] Setting key 4768eb0e72e3e3cb5ee985a0842d3d3b
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:16:14.836197 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.824876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58047 (local address 127.9.40.126:41587)
0809 02:16:14.824926 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.824928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.825472 (+ 544us) server_negotiation.cc:407] Connection header received
0809 02:16:14.826241 (+ 769us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.826242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.826257 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.826290 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.827176 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.827892 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.829485 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.829794 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.829846 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.832641 (+ 2795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.832643 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.832643 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.832651 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.834393 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.834976 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.834977 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.834978 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.835042 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.835363 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.835364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.835365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.835517 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.835598 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.835990 (+ 392us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.836085 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:16:14.837070 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.824885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41587 (local address 127.9.40.67:58047)
0809 02:16:14.825329 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.825381 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.826101 (+ 720us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.826492 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.826497 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.826777 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:16:14.827085 (+ 308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.827092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.828006 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.828008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.829357 (+ 1349us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.829360 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.829950 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.829951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.830102 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.830760 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:16:14.830772 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:16:14.832520 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:16:14.834491 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.834499 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.834501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.834867 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.835124 (+ 257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:16:14.835125 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:16:14.835125 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:16:14.835270 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:16:14.835605 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:16:14.835611 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:16:14.835805 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.836604 (+ 799us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.836822 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"thread_start_us":101,"threads_started":1}
I20260809 02:16:14.837572 22283 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41587
I20260809 02:16:14.837685 22283 heartbeater.cc:464] Registering TS with master...
I20260809 02:16:14.837921 22283 heartbeater.cc:511] Master 127.9.40.126:41587 requested a full tablet report, sending...
I20260809 02:16:14.838347 21751 ts_manager.cc:194] Registered new tserver with Master: 7b2b302e54b749589068dc6dbfc6399c (127.9.40.67:36753)
I20260809 02:16:14.838860 21751 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:58047
I20260809 02:16:14.850075 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:16:14.858664 21985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.851662 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59324 (local address 127.9.40.126:41587)
0809 02:16:14.851734 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.851736 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.851812 (+ 76us) server_negotiation.cc:407] Connection header received
0809 02:16:14.851860 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.851861 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.851872 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.851954 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:16:14.852544 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.853290 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.854067 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.854355 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.854431 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.855868 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:16:14.855869 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:16:14.855869 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:16:14.855878 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:16:14.857530 (+ 1652us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.857866 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.857867 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.857867 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.857932 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:16:14.858160 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:16:14.858161 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:16:14.858161 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:16:14.858297 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:16:14.858399 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.858488 (+ 89us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.858568 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:16:14.860569 21751 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:16:14.863013 21751 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:16:14.870762 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.866053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48036 (local address 127.9.40.65:43195)
0809 02:16:14.866350 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.866355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.866365 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:16:14.866404 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.866406 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.866419 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.866482 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:14.866670 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.867536 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.869066 (+ 1530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870134 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.870193 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.870265 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.870609 (+ 344us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.870673 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":269,"thread_start_us":56,"threads_started":1}
I20260809 02:16:14.872886 22299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.865780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43195 (local address 127.0.0.1:48036)
0809 02:16:14.866099 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.866142 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.866209 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.866512 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.866515 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.866517 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:14.866595 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.866597 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.867615 (+ 1018us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.867615 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.868925 (+ 1310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.868928 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870338 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.870340 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.870446 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.870463 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.870532 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.872792 (+ 2260us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":2191,"thread_start_us":94,"threads_started":1}
I20260809 02:16:14.873068 22296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.865730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:41141 (local address 127.0.0.1:49680)
0809 02:16:14.866182 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.866194 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.866251 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.869066 (+ 2815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.869067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.869069 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:14.869124 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.869125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870434 (+ 1309us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.870435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.871609 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.871612 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.872755 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.872756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.872833 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.872838 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.872899 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.872948 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260809 02:16:14.873275 22297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.865961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36753 (local address 127.0.0.1:42034)
0809 02:16:14.869137 (+ 3176us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:14.869147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:14.869197 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:14.869472 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:14.869473 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:14.869475 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:14.869630 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.869632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870912 (+ 1280us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.870913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.871936 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:14.871938 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.872958 (+ 1020us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.872959 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:14.873041 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.873045 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:16:14.873093 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:16:14.873132 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3154,"thread_start_us":77,"threads_started":1}
I20260809 02:16:14.873541 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.866182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.9.40.67:36753)
0809 02:16:14.866580 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.866583 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.869260 (+ 2677us) server_negotiation.cc:407] Connection header received
0809 02:16:14.869307 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.869310 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.869324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.869394 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:14.869714 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870824 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.872042 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.872788 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.872848 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.872922 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.873414 (+ 492us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.873454 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":87,"threads_started":1}
I20260809 02:16:14.873862 21900 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 (DEFAULT_TABLE table=test-table [id=6af233137f9c47e3a85476c0deca0f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:14.874243 21900 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1d3682e36f45aeb9ac30ff01e46929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:14.874250 22203 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 (DEFAULT_TABLE table=test-table [id=6af233137f9c47e3a85476c0deca0f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:14.874670 22203 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1d3682e36f45aeb9ac30ff01e46929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:14.876858 22303 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: Bootstrap starting.
I20260809 02:16:14.877398 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:14.865840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49680 (local address 127.9.40.66:41141)
0809 02:16:14.868850 (+ 3010us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:14.868855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:14.868866 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:16:14.868918 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:14.868921 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:14.868938 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:14.869019 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:14.869244 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.870333 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.871819 (+ 1486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:14.872635 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:14.872713 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:14.877220 (+ 4507us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:14.877256 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 02:16:14.877309 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2967,"thread_start_us":83,"threads_started":1}
I20260809 02:16:14.877664 22303 tablet_bootstrap.cc:654] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:14.878032 22303 log.cc:824] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:14.878146 22063 tablet_service.cc:1467] Processing CreateTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 (DEFAULT_TABLE table=test-table [id=6af233137f9c47e3a85476c0deca0f60]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:16:14.878451 22063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a1d3682e36f45aeb9ac30ff01e46929. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:16:14.885761 22304 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: Bootstrap starting.
I20260809 02:16:14.886863 22304 tablet_bootstrap.cc:654] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:14.887259 22304 log.cc:824] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:14.888069 22304 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: No bootstrap required, opened a new log
I20260809 02:16:14.888135 22304 ts_tablet_manager.cc:1397] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:16:14.888864 22306 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Bootstrap starting.
I20260809 02:16:14.890152 22306 tablet_bootstrap.cc:654] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Neither blocks nor log segments found. Creating new log.
I20260809 02:16:14.890280 22304 raft_consensus.cc:348] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.890451 22304 raft_consensus.cc:374] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:14.890486 22304 raft_consensus.cc:729] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f801ea2e4234f8ab68e083904860b16, State: Initialized, Role: FOLLOWER
I20260809 02:16:14.890472 22306 log.cc:824] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:16:14.890591 22304 consensus_queue.cc:260] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.890836 22304 ts_tablet_manager.cc:1428] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:14.891093 22138 heartbeater.cc:503] Master 127.9.40.126:41587 was elected leader, sending a full tablet report...
I20260809 02:16:14.893394 22303 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: No bootstrap required, opened a new log
I20260809 02:16:14.893533 22303 ts_tablet_manager.cc:1397] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.000s
I20260809 02:16:14.895624 22303 raft_consensus.cc:348] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.895797 22303 raft_consensus.cc:374] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:14.895844 22303 raft_consensus.cc:729] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd7ea5fb0b074e64bb9b8c47412f306f, State: Initialized, Role: FOLLOWER
I20260809 02:16:14.896010 22303 consensus_queue.cc:260] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.896301 22303 ts_tablet_manager.cc:1428] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:16:14.896463 21981 heartbeater.cc:503] Master 127.9.40.126:41587 was elected leader, sending a full tablet report...
I20260809 02:16:14.896946 22306 tablet_bootstrap.cc:492] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: No bootstrap required, opened a new log
I20260809 02:16:14.897027 22306 ts_tablet_manager.cc:1397] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Time spent bootstrapping tablet: real 0.008s user 0.000s sys 0.002s
I20260809 02:16:14.899164 22306 raft_consensus.cc:348] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.899363 22306 raft_consensus.cc:374] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:16:14.899406 22306 raft_consensus.cc:729] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7b2b302e54b749589068dc6dbfc6399c, State: Initialized, Role: FOLLOWER
I20260809 02:16:14.899536 22306 consensus_queue.cc:260] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:14.899828 22306 ts_tablet_manager.cc:1428] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260809 02:16:14.899972 22283 heartbeater.cc:503] Master 127.9.40.126:41587 was elected leader, sending a full tablet report...
Aug 09 02:16:14 dist-test-slave-qg60 krb5kdc[21237](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241774, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:15.024514 22311 raft_consensus.cc:482] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:15.024672 22311 raft_consensus.cc:504] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:15.025655 22311 leader_election.cc:288] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f801ea2e4234f8ab68e083904860b16 (127.9.40.66:41141), bd7ea5fb0b074e64bb9b8c47412f306f (127.9.40.65:43195)
I20260809 02:16:15.026574 22310 raft_consensus.cc:482] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:16:15.026692 22310 raft_consensus.cc:504] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:15.027032 22310 leader_election.cc:288] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1f801ea2e4234f8ab68e083904860b16 (127.9.40.66:41141), 7b2b302e54b749589068dc6dbfc6399c (127.9.40.67:36753)
I20260809 02:16:15.030048 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.026287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45689 (local address 127.9.40.66:41141)
0809 02:16:15.026482 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:15.026484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:15.026492 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:16:15.026749 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:15.026751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:15.026760 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:15.026790 (+ 30us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:15.027053 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.027755 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.029199 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.029722 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.029760 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.029768 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:15.029946 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:16:15.029980 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170}
I20260809 02:16:15.030082 22312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.025889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:41141 (local address 127.9.40.67:45689)
0809 02:16:15.026241 (+ 352us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.026254 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:15.026641 (+ 387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:15.026860 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:15.026860 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:15.026862 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:15.026954 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.026956 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.027916 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.027917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.029087 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.029090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.029817 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.029817 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.029870 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.029873 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:16:15.029940 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:16:15.029980 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":114,"threads_started":1}
I20260809 02:16:15.030260 22285 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.025860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43195 (local address 127.9.40.67:56359)
0809 02:16:15.026038 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.026050 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:15.026102 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:15.026645 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:15.026646 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:15.026648 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:15.026719 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.026721 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.027908 (+ 1187us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.027909 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.029128 (+ 1219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.029131 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.030007 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.030008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.030089 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.030092 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:16:15.030140 (+ 48us) client_negotiation.cc:240] Negotiation successful
0809 02:16:15.030176 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156}
I20260809 02:16:15.030620 22084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "7b2b302e54b749589068dc6dbfc6399c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f801ea2e4234f8ab68e083904860b16" is_pre_election: true
I20260809 02:16:15.030789 22300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.026217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:56359 (local address 127.9.40.65:43195)
0809 02:16:15.026331 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:15.026333 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:15.026341 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:16:15.026368 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:15.026369 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:15.026378 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:15.026424 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:15.026871 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.027603 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.029272 (+ 1669us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.029863 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.029928 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.029936 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:15.030682 (+ 746us) server_negotiation.cc:299] Negotiation successful
0809 02:16:15.030727 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260809 02:16:15.030795 22084 raft_consensus.cc:2454] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b2b302e54b749589068dc6dbfc6399c in term 0.
I20260809 02:16:15.031023 22164 leader_election.cc:302] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1f801ea2e4234f8ab68e083904860b16, 7b2b302e54b749589068dc6dbfc6399c; no voters:
I20260809 02:16:15.031066 21923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "7b2b302e54b749589068dc6dbfc6399c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" is_pre_election: true
I20260809 02:16:15.031160 21923 raft_consensus.cc:2454] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 7b2b302e54b749589068dc6dbfc6399c in term 0.
I20260809 02:16:15.031145 22311 raft_consensus.cc:2781] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:16:15.031210 22311 raft_consensus.cc:482] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:16:15.031231 22311 raft_consensus.cc:3037] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:15.032215 22311 raft_consensus.cc:504] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:15.032347 22311 leader_election.cc:288] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [CANDIDATE]: Term 1 election: Requested vote from peers 1f801ea2e4234f8ab68e083904860b16 (127.9.40.66:41141), bd7ea5fb0b074e64bb9b8c47412f306f (127.9.40.65:43195)
I20260809 02:16:15.032600 21923 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "7b2b302e54b749589068dc6dbfc6399c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f"
I20260809 02:16:15.032680 21923 raft_consensus.cc:3037] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:15.033620 21923 raft_consensus.cc:2454] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b2b302e54b749589068dc6dbfc6399c in term 1.
I20260809 02:16:15.033812 22162 leader_election.cc:302] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7b2b302e54b749589068dc6dbfc6399c, bd7ea5fb0b074e64bb9b8c47412f306f; no voters:
I20260809 02:16:15.033969 22084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "7b2b302e54b749589068dc6dbfc6399c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f801ea2e4234f8ab68e083904860b16"
I20260809 02:16:15.034068 22084 raft_consensus.cc:3037] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:16:15.035025 22084 raft_consensus.cc:2454] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 7b2b302e54b749589068dc6dbfc6399c in term 1.
I20260809 02:16:15.035281 22311 raft_consensus.cc:2781] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:16:15.035456 22311 raft_consensus.cc:686] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 1 LEADER]: Becoming Leader. State: Replica: 7b2b302e54b749589068dc6dbfc6399c, State: Running, Role: LEADER
I20260809 02:16:15.035552 22311 consensus_queue.cc:237] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } }
I20260809 02:16:15.036482 21751 catalog_manager.cc:5495] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c reported cstate change: term changed from 0 to 1, leader changed from <none> to 7b2b302e54b749589068dc6dbfc6399c (127.9.40.67). New cstate: current_term: 1 leader_uuid: "7b2b302e54b749589068dc6dbfc6399c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7b2b302e54b749589068dc6dbfc6399c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36753 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 } health_report { overall_health: UNKNOWN } } }
I20260809 02:16:15.037588 22298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.027736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:38503 (local address 127.9.40.66:41141)
0809 02:16:15.030133 (+ 2397us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:15.030135 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:15.030143 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:16:15.030164 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:15.030165 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:15.030176 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:15.030224 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:15.030759 (+ 535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.032055 (+ 1296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.036067 (+ 4012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.036555 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.036604 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.036615 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:15.037459 (+ 844us) server_negotiation.cc:299] Negotiation successful
0809 02:16:15.037520 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2382,"thread_start_us":85,"threads_started":1}
I20260809 02:16:15.037962 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.027230 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:41141 (local address 127.9.40.65:38503)
0809 02:16:15.029277 (+ 2047us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.029288 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:15.029332 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:15.030523 (+ 1191us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:15.030524 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:15.030526 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:15.030616 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.030618 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.032142 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.032143 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.035062 (+ 2919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.035066 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.036934 (+ 1868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.036935 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.037015 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.037019 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:16:15.037821 (+ 802us) client_negotiation.cc:240] Negotiation successful
0809 02:16:15.037876 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2029,"thread_start_us":83,"threads_started":1}
I20260809 02:16:15.039322 22084 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1f801ea2e4234f8ab68e083904860b16" is_pre_election: true
I20260809 02:16:15.039433 22084 raft_consensus.cc:2379] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate bd7ea5fb0b074e64bb9b8c47412f306f in current term 1: Already voted for candidate 7b2b302e54b749589068dc6dbfc6399c in this term.
I20260809 02:16:15.042969 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.027459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:42371 (local address 127.9.40.67:36753)
0809 02:16:15.027505 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:15.027507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:15.039384 (+ 11877us) server_negotiation.cc:407] Connection header received
0809 02:16:15.039991 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:15.039992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:15.040003 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:15.040036 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:16:15.040232 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.040884 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.042088 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.042587 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.042634 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.042644 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:15.042847 (+ 203us) server_negotiation.cc:299] Negotiation successful
0809 02:16:15.042895 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:16:15.043256 22314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.027408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36753 (local address 127.9.40.65:42371)
0809 02:16:15.039327 (+ 11919us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.039339 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:15.039411 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:15.040104 (+ 693us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:16:15.040105 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:16:15.040107 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:16:15.040147 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.040148 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.040976 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.040978 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.041963 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:16:15.041966 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.042705 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.042706 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:16:15.042758 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.042761 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:16:15.043131 (+ 370us) client_negotiation.cc:240] Negotiation successful
0809 02:16:15.043178 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11904,"thread_start_us":53,"threads_started":1}
I20260809 02:16:15.043728 22234 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a1d3682e36f45aeb9ac30ff01e46929" candidate_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7b2b302e54b749589068dc6dbfc6399c" is_pre_election: true
I20260809 02:16:15.044073 21857 leader_election.cc:302] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bd7ea5fb0b074e64bb9b8c47412f306f; no voters: 1f801ea2e4234f8ab68e083904860b16, 7b2b302e54b749589068dc6dbfc6399c
I20260809 02:16:15.044302 22310 raft_consensus.cc:2726] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260809 02:16:15.062587 22302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.059329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42038 (local address 127.9.40.67:36753)
0809 02:16:15.059376 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:16:15.059379 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:16:15.059387 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:16:15.059431 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:16:15.059432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:16:15.059454 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:16:15.059550 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:16:15.059672 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.060418 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.061644 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:16:15.061939 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:16:15.062026 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:16:15.062196 (+ 170us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:16:15.062340 (+ 144us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:16:15.062434 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:16:15.062465 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 02:16:15.062492 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:16:15.064826 21923 raft_consensus.cc:1261] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 1 FOLLOWER]: Refusing update from remote peer 7b2b302e54b749589068dc6dbfc6399c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:15.064888 22084 raft_consensus.cc:1261] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16 [term 1 FOLLOWER]: Refusing update from remote peer 7b2b302e54b749589068dc6dbfc6399c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:16:15.065404 22316 consensus_queue.cc:1035] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [LEADER]: Connected to new peer: Peer: permanent_uuid: "1f801ea2e4234f8ab68e083904860b16" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 41141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:15.065655 22311 consensus_queue.cc:1035] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [LEADER]: Connected to new peer: Peer: permanent_uuid: "bd7ea5fb0b074e64bb9b8c47412f306f" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 43195 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:16:15.072521 22323 mvcc.cc:205] Tried to move back new op lower bound from 7316446310662717440 to 7316446310546243584. Current Snapshot: MvccSnapshot[applied={T|T < 7316446310662717440}]
I20260809 02:16:15.072928 22321 mvcc.cc:205] Tried to move back new op lower bound from 7316446310662717440 to 7316446310546243584. Current Snapshot: MvccSnapshot[applied={T|T < 7316446310662717440}]
I20260809 02:16:15.073092 21751 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:15.073187 21751 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:59324:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:16:15.073877 21751 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8660caf0279c4188a0ae318605fed204: Sending DeleteTablet for 3 replicas of tablet 6a1d3682e36f45aeb9ac30ff01e46929
I20260809 02:16:15.074211 22063 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:15 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49680
I20260809 02:16:15.074250 22203 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:15 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42034
I20260809 02:16:15.074599 21900 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a1d3682e36f45aeb9ac30ff01e46929 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:16:15 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48036
I20260809 02:16:15.074820 22331 tablet_replica.cc:330] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f: stopping tablet replica
I20260809 02:16:15.074925 22331 raft_consensus.cc:2229] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:16:15.074995 22331 raft_consensus.cc:2258] T 6a1d3682e36f45aeb9ac30ff01e46929 P bd7ea5fb0b074e64bb9b8c47412f306f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:15.075176 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 21835
W20260809 02:16:15.087749 21732 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:43195 (error 108)
I20260809 02:16:15.088702 22299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:16:15.088580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:43195 (local address 127.0.0.1:48048)
0809 02:16:15.088636 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.088666 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43195: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:16:15.089079 22330 tablet_replica.cc:330] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: stopping tablet replica
I20260809 02:16:15.088860 22329 tablet_replica.cc:330] T 6a1d3682e36f45aeb9ac30ff01e46929 P 1f801ea2e4234f8ab68e083904860b16: stopping tablet replica
W20260809 02:16:15.089236 21732 catalog_manager.cc:4584] TS bd7ea5fb0b074e64bb9b8c47412f306f (127.9.40.65:43195): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a1d3682e36f45aeb9ac30ff01e46929: Network error: Client connection negotiation failed: client connection to 127.9.40.65:43195: connect: Connection refused (error 111)
I20260809 02:16:15.089288 22330 raft_consensus.cc:2229] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:16:15.089442 22330 raft_consensus.cc:2258] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:16:15.089509 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 21988
I20260809 02:16:15.092607 22330 ts_tablet_manager.cc:1905] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:16:15.094841 22330 ts_tablet_manager.cc:1918] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:16:15.094978 22330 log.cc:1196] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/6a1d3682e36f45aeb9ac30ff01e46929
I20260809 02:16:15.095342 22330 ts_tablet_manager.cc:1939] T 6a1d3682e36f45aeb9ac30ff01e46929 P 7b2b302e54b749589068dc6dbfc6399c: Deleting consensus metadata
I20260809 02:16:15.095861 21734 catalog_manager.cc:4841] TS 7b2b302e54b749589068dc6dbfc6399c (127.9.40.67:36753): tablet 6a1d3682e36f45aeb9ac30ff01e46929 (table test-table [id=6af233137f9c47e3a85476c0deca0f60]) successfully deleted
W20260809 02:16:15.113953 22299 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:16:15.113634 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:41141 (local address 127.0.0.1:49690)
0809 02:16:15.113686 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:16:15.113697 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:16:15.113739 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:16:15.113892 (+ 153us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:41141: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 02:16:15.114612 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 22144
I20260809 02:16:15.120241 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 21716
I20260809 02:16:15.128711 9377 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:16:15.473357 9377 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:16:15.473479 9377 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:16:15.818574 9377 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:16:15.818 UTC [21270] LOG: received smart shutdown request
2026-08-09 02:16:15.823 UTC [21270] LOG: background worker "logical replication launcher" (PID 21280) exited with exit code 1
2026-08-09 02:16:15.823 UTC [21274] LOG: shutting down
2026-08-09 02:16:16.015 UTC [21270] LOG: database system is shut down
2026-08-09T02:16:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (48478 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:16:16 dist-test-slave-qg60 krb5kdc[22341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:16:16 dist-test-slave-qg60 krb5kdc[22341](info): set up 2 sockets
Aug 09 02:16:16 dist-test-slave-qg60 krb5kdc[22341](info): commencing operation
krb5kdc: starting...
W20260809 02:16:17.251235 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.100s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:16:17 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241777, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:16:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:16:17Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangeradmin_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangerlookup_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
I20260809 02:16:17.439627 9377 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskQln13t/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/postgres -l logfile start
2026-08-09 02:16:20.117 UTC [22373] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-09 02:16:20.117 UTC [22373] LOG: listening on IPv4 address "127.9.40.84", port 57427
2026-08-09 02:16:20.120 UTC [22373] LOG: listening on Unix socket "/tmp/.s.PGSQL.57427"
2026-08-09 02:16:20.142 UTC [22376] LOG: database system was shut down at 2026-08-09 02:16:18 UTC
2026-08-09 02:16:20.146 UTC [22373] LOG: database system is ready to accept connections
I20260809 02:16:21.200691 9377 mini_postgres.cc:96] Postgres bound to 57427
2026-08-09 02:16:21.207 UTC [22385] FATAL: database "slave" does not exist
127.9.40.84:57427 - accepting connections
I20260809 02:16:21.208200 9377 mini_ranger.cc:161] Starting Ranger...
I20260809 02:16:21.223894 9377 mini_ranger.cc:84] Created miniranger Postgres user
I20260809 02:16:21.771306 9377 mini_ranger.cc:87] Created ranger Postgres database
I20260809 02:16:21.771396 9377 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin
2026-08-09 02:16:22,033 [I] DB FLAVOR :POSTGRES
2026-08-09 02:16:22,034 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:16:22,034 [I] Checking connection..
2026-08-09 02:16:22,034 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-09 02:16:22,309 [I] Checking connection passed.
2026-08-09 02:16:22,309 [I] --------- Verifying version history table ---------
2026-08-09 02:16:22,309 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:22,584 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:16:22,585 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:22,852 [I] Table x_db_version_h does not exist in database ranger
2026-08-09 02:16:22,853 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-09 02:16:22,853 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-09 02:16:23.099 UTC [22456] WARNING: there is no transaction in progress
2026-08-09 02:16:23,108 [I] create_dbversion_catalog.sql file imported successfully
2026-08-09 02:16:23,108 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-09 02:16:23,350 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-09 02:16:23,350 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:16:23,351 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-09 02:16:23,632 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-09 02:16:23,909 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-qg60.c.gcp-upstream.internal','N') ;"
2026-08-09 02:16:24,170 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:16:24,413 [I] Table x_portal_user does not exist in database ranger
2026-08-09 02:16:24,414 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:16:24,674 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-09 02:16:24,675 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:16:24,916 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:16:25,167 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-09 02:16:25,168 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-09 02:16:26.220 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.240 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.259 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.277 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.299 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.318 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.378 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.386 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.402 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.413 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.424 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.440 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.451 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.458 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.466 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:26.474 UTC [22571] WARNING: there is no transaction in progress
2026-08-09 02:16:27,206 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-09 02:16:27,207 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-09 02:16:27,467 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-09 02:16:27,467 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-09 02:16:27,751 [I] Table x_portal_user already exists in database 'ranger'
2026-08-09 02:16:27,751 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-09 02:16:28,010 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-09 02:16:28,011 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-09 02:16:28,288 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-09 02:16:28,563 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-qg60.c.gcp-upstream.internal';"
2026-08-09 02:16:28,827 [I] CORE_DB_SCHEMA import status has been updated
2026-08-09 02:16:28,827 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-09 02:16:29,068 [I] DB_PATCHES have already been applied
I20260809 02:16:29.073896 9377 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskQln13t/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260809 02:16:29.073989 9377 mini_ranger.cc:193] Using host: 127.9.40.84
I20260809 02:16:29.077783 9377 mini_ranger.cc:227] Ranger admin URL: http://127.9.40.84:49167
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-admin/ews/webapp].
Aug 09, 2026 2:16:29 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260809 02:16:30.326792 9377 mini_ranger.cc:160] Time spent starting Ranger: real 9.119s user 0.000s sys 0.011s
Aug 09, 2026 2:16:50 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 09, 2026 2:16:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 09, 2026 2:16:50 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 09, 2026 2:16:50 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 09, 2026 2:16:51 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260809 02:16:52.196171 9377 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.9.40.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.9.40.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.9.40.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/rangerkms_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.9.40.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.9.40.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.9.40.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/HTTP_127.9.40.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 09 02:16:52 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241812, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260809 02:16:52.292963 9377 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260809 02:16:52.304078 9377 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260809 02:16:53.352056 9377 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260809 02:16:53.352149 9377 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms
I20260809 02:16:53.352164 9377 mini_ranger_kms.cc:227] Using postgres at 127.9.40.84:57427
2026-08-09 02:16:53,561 [I] DB FLAVOR :POSTGRES
2026-08-09 02:16:53,561 [I] --------- Verifying Ranger DB connection ---------
2026-08-09 02:16:53,561 [I] Checking connection
2026-08-09 02:16:53,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-09 02:16:53,838 [I] connection success
2026-08-09 02:16:53,838 [I] --------- Verifying Ranger DB tables ---------
2026-08-09 02:16:53,838 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-09 02:16:53,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-09 02:16:54,120 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-09 02:16:54,120 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-09 02:16:54,121 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-09 02:16:54,121 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.9.40.84:57427/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-09 02:16:54,388 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260809 02:16:54.855618 9377 mini_ranger_kms.cc:310] Created kms service
I20260809 02:16:55.049088 9377 mini_ranger_kms.cc:326] Created kudu user
I20260809 02:16:55.100543 9377 mini_ranger_kms.cc:343] Created rangerkms user
Aug 09 02:16:55 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241812, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:16:55.536167 9377 mini_ranger_kms.cc:384] Added ranger policy
I20260809 02:16:55.536291 9377 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms:/tmp/dist-test-taskQln13t/build/release/bin/postgresql.jar:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskQln13t/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260809 02:16:55.536311 9377 mini_ranger_kms.cc:240] Using host: 127.9.40.84
I20260809 02:16:55.540385 9377 mini_ranger_kms.cc:276] Ranger KMS PID: 22769
I20260809 02:16:55.540483 9377 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.9.40.84:58325
Aug 09, 2026 2:16:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 09, 2026 2:16:56 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 09, 2026 2:16:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 09, 2026 2:16:56 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 09, 2026 2:16:57 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-kms/ews/webapp].
Aug 09, 2026 2:16:57 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 09 02:16:57 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241817, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:16:57.938616 9377 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.646s user 0.007s sys 0.182s
I20260809 02:16:57.938742 9377 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260809 02:16:57.938804 9377 mini_ranger_kms.cc:401] 127.9.40.84:58325/kms/v1/keys
Aug 09 02:16:58 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241817, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 09 02:16:58 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241817, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.9.40.84@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
Aug 09 02:17:01 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:17:01.148486 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43747
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43747
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:01.234527 22825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:01.234776 22825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:01.234802 22825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:01.236012 22825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:01.236063 22825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:01.236078 22825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:01.236091 22825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:01.236104 22825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:01.237919 22825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=127.9.40.84:58325/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43747
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:01.238256 22825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:01.238511 22825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:17:01.244231 22825 server_base.cc:1034] running on GCE node
W20260809 02:17:01.244349 22835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:01.244344 22830 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:01.244659 22831 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:01.244952 22825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:01.245306 22825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:01.246500 22825 hybrid_clock.cc:648] HybridClock initialized: now 1786241821246479 us; error 49 us; skew 500 ppm
Aug 09 02:17:01 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241821, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:01.253059 22825 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:01.253633 22825 webserver.cc:460] Webserver started at http://127.9.40.126:39533/ using document root <none> and password file <none>
I20260809 02:17:01.253881 22825 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:01.253965 22825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:01.254153 22825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:01 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241821, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:01.317589 22825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3"
format_stamp: "Formatted at 2026-08-09 02:17:01 on dist-test-slave-qg60"
server_key: "69f0ebf41add8e1f4a34ff64c6615be0"
server_key_iv: "83f223b4f68d47ba49e6cc286d9737a5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:01.318039 22825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3"
format_stamp: "Formatted at 2026-08-09 02:17:01 on dist-test-slave-qg60"
server_key: "69f0ebf41add8e1f4a34ff64c6615be0"
server_key_iv: "83f223b4f68d47ba49e6cc286d9737a5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:01.319960 22825 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:01.335835 22843 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:01.336201 22825 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:01.336321 22825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3"
format_stamp: "Formatted at 2026-08-09 02:17:01 on dist-test-slave-qg60"
server_key: "69f0ebf41add8e1f4a34ff64c6615be0"
server_key_iv: "83f223b4f68d47ba49e6cc286d9737a5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:01.336426 22825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:01.351146 22825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:01.353420 22825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:01.353574 22825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:01.358850 22825 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43747
I20260809 02:17:01.358884 22902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43747 every 8 connection(s)
I20260809 02:17:01.359318 22825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:01.360059 22903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:01.362432 22903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Bootstrap starting.
I20260809 02:17:01.363047 22903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:01.363360 22903 log.cc:824] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:01.364107 22903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: No bootstrap required, opened a new log
I20260809 02:17:01.365860 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 22825
I20260809 02:17:01.365962 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:17:01.366189 9377 external_mini_cluster.cc:1428] Setting key 43dac1de30f7a435601ed54eec4b71ca
I20260809 02:17:01.366452 22903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:01.366705 22903 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:01.366770 22903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6f9c2e2ce04316a9314ed7c3226eb3, State: Initialized, Role: FOLLOWER
I20260809 02:17:01.366942 22903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:01.367055 22903 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:01.367100 22903 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:01.367139 22903 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:01.367905 22903 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:01.368032 22903 leader_election.cc:302] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f6f9c2e2ce04316a9314ed7c3226eb3; no voters:
I20260809 02:17:01.368230 22903 leader_election.cc:288] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:17:01.368263 22908 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:01.368484 22908 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 LEADER]: Becoming Leader. State: Replica: 3f6f9c2e2ce04316a9314ed7c3226eb3, State: Running, Role: LEADER
I20260809 02:17:01.368590 22903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:01.368613 22908 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:01.369024 22910 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } } }
I20260809 02:17:01.369195 22910 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:01.369410 22909 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f6f9c2e2ce04316a9314ed7c3226eb3. Latest consensus state: current_term: 1 leader_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } } }
I20260809 02:17:01.369544 22909 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:01.369719 22912 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:01.370321 22912 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:01.370652 22903 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
Aug 09 02:17:01 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241821, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:01.378063 22912 catalog_manager.cc:1347] Generated new cluster ID: 88dc9a88ba344b5ebfbeb12f215f984c
I20260809 02:17:01.378206 22912 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:01.381906 22907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:01.366884 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38998 (local address 127.9.40.126:43747)
0809 02:17:01.367248 (+ 364us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:01.367257 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:01.367291 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:17:01.367403 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:01.367410 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:01.367488 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:01.367641 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:01.368444 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:01.369984 (+ 1540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:01.371078 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:01.374126 (+ 3048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:01.374327 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:01.377752 (+ 3425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:01.377756 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:01.377756 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:01.377769 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:01.380033 (+ 2264us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:01.380461 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:01.380464 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:01.380464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:01.380579 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:01.380816 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:01.380817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:01.380817 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:01.381151 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:01.381268 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:01.381527 (+ 259us) server_negotiation.cc:299] Negotiation successful
0809 02:17:01.381668 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":94,"threads_started":1}
I20260809 02:17:01.435207 22912 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:01.436053 22912 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:01.443719 22912 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Generated new TSK 0
I20260809 02:17:01.444008 22912 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:17:02.201308 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:02.328469 22950 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:02.328766 22950 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:02.328802 22950 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:02.330575 22950 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:02.330641 22950 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:02.330679 22950 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:02.333000 22950 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22950
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:02.333417 22950 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:02.333700 22950 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:02.339996 22958 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:02.340071 22956 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:02.340610 22955 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:02.341763 22950 server_base.cc:1034] running on GCE node
I20260809 02:17:02.341975 22950 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:02.342239 22950 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:02.343392 22950 hybrid_clock.cc:648] HybridClock initialized: now 1786241822343371 us; error 34 us; skew 500 ppm
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:02.346740 22950 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:02.347329 22950 webserver.cc:460] Webserver started at http://127.9.40.65:35489/ using document root <none> and password file <none>
I20260809 02:17:02.347631 22950 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:02.347702 22950 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:02.347901 22950 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:02.363983 22950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "b82c67a843044dc095347b7f43c8b180"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "de509e1a8f69e5d0b79acf296a61471b"
server_key_iv: "81092b6046609965a8f87cbaea7cbfce"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.364557 22950 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "b82c67a843044dc095347b7f43c8b180"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "de509e1a8f69e5d0b79acf296a61471b"
server_key_iv: "81092b6046609965a8f87cbaea7cbfce"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.366647 22950 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.001s
I20260809 02:17:02.381997 22966 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.382181 22950 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:02.382264 22950 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "b82c67a843044dc095347b7f43c8b180"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "de509e1a8f69e5d0b79acf296a61471b"
server_key_iv: "81092b6046609965a8f87cbaea7cbfce"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.382359 22950 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:02.427336 22950 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:02.429963 22950 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:02.430194 22950 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:02.430471 22950 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:02.430873 22950 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:02.430958 22950 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.431072 22950 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:02.431144 22950 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.438661 22950 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:41319
I20260809 02:17:02.439190 22950 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:02.441437 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 22950
I20260809 02:17:02.441547 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:17:02.441771 9377 external_mini_cluster.cc:1428] Setting key f47ab430a543cffa9db0e503404b6d31
I20260809 02:17:02.440826 23091 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:41319 every 8 connection(s)
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:02.461736 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:02.442468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36009 (local address 127.9.40.126:43747)
0809 02:17:02.444880 (+ 2412us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:02.444882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:02.449069 (+ 4187us) server_negotiation.cc:407] Connection header received
0809 02:17:02.450041 (+ 972us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:02.450043 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:02.450063 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:02.450114 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:02.451085 (+ 971us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.451966 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.453619 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.453841 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.453937 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:02.457570 (+ 3633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:02.457572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:02.457572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:02.457581 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:02.459670 (+ 2089us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:02.460225 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:02.460226 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:02.460227 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:02.460296 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:02.460754 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:02.460755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:02.460756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:02.460930 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:02.461067 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:02.461471 (+ 404us) server_negotiation.cc:299] Negotiation successful
0809 02:17:02.461576 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2380,"thread_start_us":88,"threads_started":1}
I20260809 02:17:02.462672 23094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:02.442219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.65:36009)
0809 02:17:02.448845 (+ 6626us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:02.448937 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:02.449875 (+ 938us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:02.450344 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:02.450350 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:02.450642 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:02.450954 (+ 312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:02.450962 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.452107 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.452108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:02.453488 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:02.453491 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.453963 (+ 472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.453964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:02.454165 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:02.454807 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:02.454816 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:02.457413 (+ 2597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:02.459785 (+ 2372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:02.459792 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:02.459795 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:02.460121 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:02.460371 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:02.460371 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:02.460372 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:02.460626 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:02.461107 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:02.461114 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:02.461308 (+ 194us) client_negotiation.cc:769] Sending connection context
0809 02:17:02.462187 (+ 879us) client_negotiation.cc:240] Negotiation successful
0809 02:17:02.462379 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6562,"thread_start_us":100,"threads_started":1}
I20260809 02:17:02.463483 23092 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:02.463603 23092 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:02.463815 23092 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:02.464485 22862 ts_manager.cc:194] Registered new tserver with Master: b82c67a843044dc095347b7f43c8b180 (127.9.40.65:41319)
I20260809 02:17:02.465449 22862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36009
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:17:02.489866 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260809 02:17:02.618711 23098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:02.619119 23098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:02.619216 23098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:02.621037 23098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:02.621166 23098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:02.621269 23098 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:02.623700 23098 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:02.624166 23098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:02.624506 23098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:02.631111 23106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:02.631395 23109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:02.632550 23098 server_base.cc:1034] running on GCE node
W20260809 02:17:02.632707 23107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:02.633062 23098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:02.633353 23098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:02.636647 23098 hybrid_clock.cc:648] HybridClock initialized: now 1786241822636621 us; error 35 us; skew 500 ppm
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:02.640092 23098 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:02.640689 23098 webserver.cc:460] Webserver started at http://127.9.40.66:39671/ using document root <none> and password file <none>
I20260809 02:17:02.641067 23098 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:02.641140 23098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:02.641319 23098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:02.657512 23098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "f211f0221d7e4396b6287dd236587050"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "d63c5d24eabfe1d671dc4511fa416162"
server_key_iv: "ccf79b776226f683e99ca8df09802c52"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.658089 23098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "f211f0221d7e4396b6287dd236587050"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "d63c5d24eabfe1d671dc4511fa416162"
server_key_iv: "ccf79b776226f683e99ca8df09802c52"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.662177 23098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.000s
I20260809 02:17:02.677982 23116 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.678254 23098 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:17:02.678359 23098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "f211f0221d7e4396b6287dd236587050"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "d63c5d24eabfe1d671dc4511fa416162"
server_key_iv: "ccf79b776226f683e99ca8df09802c52"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.678452 23098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:02.701339 23098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:02.704034 23098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:02.704262 23098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:02.704582 23098 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:02.705060 23098 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:02.705158 23098 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.705240 23098 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:02.705305 23098 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.712236 23098 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:34561
I20260809 02:17:02.712802 23098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:02.714993 23242 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:34561 every 8 connection(s)
I20260809 02:17:02.719933 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23098
I20260809 02:17:02.720049 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:17:02.720259 9377 external_mini_cluster.cc:1428] Setting key fc16770ec095cbfc5bf66f3bd06b4b48
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:02.727233 23245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:02.715752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.66:48443)
0809 02:17:02.715985 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:02.716036 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:02.716676 (+ 640us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:02.717114 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:02.717118 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:02.717404 (+ 286us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:02.717689 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:02.717696 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.718737 (+ 1041us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.718737 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:02.720005 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:02.720008 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.720580 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.720581 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:02.720719 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:02.721400 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:02.721410 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:02.723182 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:02.725294 (+ 2112us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:02.725302 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:02.725304 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:02.725644 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:02.725930 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:02.725931 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:02.725931 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:02.726110 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:02.726446 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:02.726452 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:02.726655 (+ 203us) client_negotiation.cc:769] Sending connection context
0809 02:17:02.726771 (+ 116us) client_negotiation.cc:240] Negotiation successful
0809 02:17:02.726955 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":108,"threads_started":1}
I20260809 02:17:02.727358 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:02.716074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:48443 (local address 127.9.40.126:43747)
0809 02:17:02.716139 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:02.716142 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:02.716154 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:17:02.716854 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:02.716856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:02.716870 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:02.716905 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:02.717816 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.718607 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.720117 (+ 1510us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:02.720418 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:02.720469 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:02.723304 (+ 2835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:02.723306 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:02.723306 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:02.723314 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:02.725178 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:02.725751 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:02.725753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:02.725754 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:02.725827 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:02.726197 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:02.726198 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:02.726198 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:02.726354 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:02.726419 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:02.727166 (+ 747us) server_negotiation.cc:299] Negotiation successful
0809 02:17:02.727265 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:17:02.731344 23243 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:02.731473 23243 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:02.731703 23243 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:02.732221 22862 ts_manager.cc:194] Registered new tserver with Master: f211f0221d7e4396b6287dd236587050 (127.9.40.66:34561)
I20260809 02:17:02.732752 22862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:48443
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:17:02.775787 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:02.917325 23249 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:02.917629 23249 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:02.917670 23249 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:02.919806 23249 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:02.919883 23249 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:02.919934 23249 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:02.922851 23249 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23249
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:02.923177 23249 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:02.923506 23249 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:02.941551 23255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:02.941990 23254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:02.943292 23257 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:02.943619 23249 server_base.cc:1034] running on GCE node
I20260809 02:17:02.943981 23249 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:02.944296 23249 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:02.945467 23249 hybrid_clock.cc:648] HybridClock initialized: now 1786241822945432 us; error 66 us; skew 500 ppm
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:02.948864 23249 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:02.949404 23249 webserver.cc:460] Webserver started at http://127.9.40.67:35033/ using document root <none> and password file <none>
I20260809 02:17:02.949692 23249 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:02.949741 23249 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:02.949867 23249 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 09 02:17:02 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:02.967168 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "e681f0baff1741c3b23dcaf19a25a251"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "ffe4e2fd49669fccd78335d5cedbd45d"
server_key_iv: "ee3b37e88b2c395081ba613994565b6e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.967674 23249 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "e681f0baff1741c3b23dcaf19a25a251"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "ffe4e2fd49669fccd78335d5cedbd45d"
server_key_iv: "ee3b37e88b2c395081ba613994565b6e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.971123 23249 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:02.992218 23265 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:02.992506 23249 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:17:02.992606 23249 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "e681f0baff1741c3b23dcaf19a25a251"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "ffe4e2fd49669fccd78335d5cedbd45d"
server_key_iv: "ee3b37e88b2c395081ba613994565b6e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:02.992708 23249 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:03.008710 23249 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:03.011565 23249 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:03.011797 23249 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:03.012115 23249 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:03.012601 23249 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:03.012698 23249 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.012825 23249 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:03.012892 23249 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.020180 23249 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:37631
I20260809 02:17:03.020761 23249 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:03.029443 23388 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:37631 every 8 connection(s)
I20260809 02:17:03.029785 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23249
I20260809 02:17:03.029906 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:17:03.030149 9377 external_mini_cluster.cc:1428] Setting key d5cec8d7634cb5e6fda91fffe4f1fe77
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241822, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:03.043792 23095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.030926 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45759 (local address 127.9.40.126:43747)
0809 02:17:03.030985 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:03.030987 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:03.031790 (+ 803us) server_negotiation.cc:407] Connection header received
0809 02:17:03.032542 (+ 752us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:03.032544 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:03.032557 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:03.032592 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:03.033561 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.034332 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.035807 (+ 1475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.036126 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.036180 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.039139 (+ 2959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:03.039140 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:03.039141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:03.039149 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:03.041171 (+ 2022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.042553 (+ 1382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.042554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.042555 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.042626 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.042938 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.042940 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.042940 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.043098 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:03.043146 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:03.043595 (+ 449us) server_negotiation.cc:299] Negotiation successful
0809 02:17:03.043697 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:17:03.044678 23391 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.030517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.67:45759)
0809 02:17:03.031646 (+ 1129us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:03.031700 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:03.032407 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:03.032843 (+ 436us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:03.032848 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:03.033140 (+ 292us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:03.033441 (+ 301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.033447 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.034462 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.034463 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.035689 (+ 1226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.035692 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.036297 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.036298 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.036442 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.037202 (+ 760us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:03.037212 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:03.039004 (+ 1792us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:03.041303 (+ 2299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.041312 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.041314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.042451 (+ 1137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.042707 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.042708 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.042709 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.042854 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.043201 (+ 347us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:03.043208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:03.043408 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 02:17:03.044199 (+ 791us) client_negotiation.cc:240] Negotiation successful
0809 02:17:03.044381 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1080,"thread_start_us":119,"threads_started":1}
I20260809 02:17:03.045408 23389 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:03.045513 23389 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.045717 23389 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:03.046190 22862 ts_manager.cc:194] Registered new tserver with Master: e681f0baff1741c3b23dcaf19a25a251 (127.9.40.67:37631)
I20260809 02:17:03.046674 22862 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:45759
I20260809 02:17:03.053149 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:03.054029 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 22950
I20260809 02:17:03.073094 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23098
I20260809 02:17:03.081679 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23249
I20260809 02:17:03.090830 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 22825
I20260809 02:17:03.101984 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43747
--webserver_interface=127.9.40.126
--webserver_port=39533
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43747
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:03.188151 23392 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:03.188375 23392 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:03.188401 23392 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:03.189673 23392 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:03.189723 23392 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:03.189738 23392 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:03.189751 23392 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:03.189764 23392 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:03.191382 23392 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43747
--ranger_config_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43747
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=39533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23392
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:03.191632 23392 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:03.191869 23392 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:03.197062 23398 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.197234 23397 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.197041 23400 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.197082 23392 server_base.cc:1034] running on GCE node
I20260809 02:17:03.197615 23392 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:03.197970 23392 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:03.199143 23392 hybrid_clock.cc:648] HybridClock initialized: now 1786241823199116 us; error 44 us; skew 500 ppm
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:03.201803 23392 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:03.202299 23392 webserver.cc:460] Webserver started at http://127.9.40.126:39533/ using document root <none> and password file <none>
I20260809 02:17:03.202559 23392 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:03.202652 23392 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:03.282820 23392 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:03.283613 23409 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.283810 23392 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:03.283910 23392 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3"
format_stamp: "Formatted at 2026-08-09 02:17:01 on dist-test-slave-qg60"
server_key: "69f0ebf41add8e1f4a34ff64c6615be0"
server_key_iv: "83f223b4f68d47ba49e6cc286d9737a5"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:03.284304 23392 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:03.292549 23392 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:03.294596 23392 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:03.294792 23392 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:03.299618 23392 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43747
I20260809 02:17:03.299664 23467 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43747 every 8 connection(s)
I20260809 02:17:03.300072 23392 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:03.301579 23468 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:17:03.302253 23468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Bootstrap starting.
I20260809 02:17:03.309888 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23392
I20260809 02:17:03.310322 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:41319
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:43747
--webserver_port=35489
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:03.311698 23468 log.cc:824] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:03.314005 23468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:03.314338 23468 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Bootstrap complete.
I20260809 02:17:03.317219 23468 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:03.317487 23468 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3f6f9c2e2ce04316a9314ed7c3226eb3, State: Initialized, Role: FOLLOWER
I20260809 02:17:03.317665 23468 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:03.317746 23468 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:03.317821 23468 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:03.317876 23468 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:17:03.319051 23468 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:03.319147 23468 leader_election.cc:302] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3f6f9c2e2ce04316a9314ed7c3226eb3; no voters:
I20260809 02:17:03.319391 23468 leader_election.cc:288] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:17:03.319432 23472 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:17:03.319566 23472 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [term 2 LEADER]: Becoming Leader. State: Replica: 3f6f9c2e2ce04316a9314ed7c3226eb3, State: Running, Role: LEADER
I20260809 02:17:03.319710 23468 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:03.319767 23472 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } }
I20260809 02:17:03.320188 23472 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3f6f9c2e2ce04316a9314ed7c3226eb3. Latest consensus state: current_term: 2 leader_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } } }
I20260809 02:17:03.320194 23473 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3f6f9c2e2ce04316a9314ed7c3226eb3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43747 } } }
I20260809 02:17:03.320339 23472 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:03.320901 23473 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:03.320884 23482 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:03.321506 23468 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260809 02:17:03.321648 23482 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:03.321856 23482 catalog_manager.cc:1259] Loaded cluster ID: 88dc9a88ba344b5ebfbeb12f215f984c
I20260809 02:17:03.321969 23482 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:03.327185 23482 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:03.327587 23482 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Loaded TSK: 0
I20260809 02:17:03.327953 23482 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:17:03.444608 23471 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:03.444989 23471 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:03.445039 23471 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:03.447041 23471 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:03.447177 23471 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:03.447252 23471 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:03.450127 23471 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:41319
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=35489
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23471
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:03.450773 23471 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:03.451184 23471 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:03.458904 23508 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.458902 23510 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.459640 23471 server_base.cc:1034] running on GCE node
W20260809 02:17:03.460906 23507 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.461202 23471 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:03.461500 23471 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:03.462675 23471 hybrid_clock.cc:648] HybridClock initialized: now 1786241823462650 us; error 46 us; skew 500 ppm
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:03.466266 23471 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:03.466912 23471 webserver.cc:460] Webserver started at http://127.9.40.65:35489/ using document root <none> and password file <none>
I20260809 02:17:03.467238 23471 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:03.467370 23471 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:03.486634 23471 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:03.488020 23518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.488212 23471 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:03.488305 23471 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "b82c67a843044dc095347b7f43c8b180"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "de509e1a8f69e5d0b79acf296a61471b"
server_key_iv: "81092b6046609965a8f87cbaea7cbfce"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:03.488970 23471 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:03.500368 23471 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:03.503002 23471 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:03.503239 23471 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:03.503539 23471 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:03.503957 23471 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:03.504046 23471 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.504171 23471 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:03.504257 23471 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.512460 23471 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:41319
I20260809 02:17:03.513100 23471 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:03.516001 23640 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:41319 every 8 connection(s)
I20260809 02:17:03.518615 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23471
I20260809 02:17:03.518958 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:34561
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:43747
--webserver_port=39671
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:03.545079 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.516992 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:36685 (local address 127.9.40.126:43747)
0809 02:17:03.517215 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:03.517227 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:03.528857 (+ 11630us) server_negotiation.cc:407] Connection header received
0809 02:17:03.530091 (+ 1234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:03.530101 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:03.530168 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:03.530340 (+ 172us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:03.531236 (+ 896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.532660 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.534501 (+ 1841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.535137 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.535254 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.538173 (+ 2919us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:03.538178 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:03.538178 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:03.538192 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:03.540285 (+ 2093us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.540902 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.540906 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.540906 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.541035 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.541355 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.541355 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.541356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.541666 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:03.541744 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:03.544622 (+ 2878us) server_negotiation.cc:299] Negotiation successful
0809 02:17:03.544820 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":102,"threads_started":1}
I20260809 02:17:03.546489 23643 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.516843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.65:36685)
0809 02:17:03.528519 (+ 11676us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:03.528609 (+ 90us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:03.529764 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:03.530426 (+ 662us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:03.530432 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:03.530726 (+ 294us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:03.531085 (+ 359us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.531092 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.532816 (+ 1724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.532817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.534387 (+ 1570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.534391 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.535384 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.535385 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.535519 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.536211 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:03.536222 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:03.538019 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:03.540388 (+ 2369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.540395 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.540397 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.540774 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.541137 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.541138 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.541138 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.541274 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.541785 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:03.541791 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:03.541947 (+ 156us) client_negotiation.cc:769] Sending connection context
0809 02:17:03.545629 (+ 3682us) client_negotiation.cc:240] Negotiation successful
0809 02:17:03.546253 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11597,"thread_start_us":109,"threads_started":1}
I20260809 02:17:03.547894 23641 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:03.548009 23641 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.548221 23641 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
W20260809 02:17:03.549198 23641 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43747 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:17:03.549268 23641 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.549326 23641 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:03.549731 23641 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.549805 23641 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
W20260809 02:17:03.550173 23641 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:17:03.643538 23645 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:03.643828 23645 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:03.643875 23645 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:03.645383 23645 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:03.645455 23645 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:03.645515 23645 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:03.647347 23645 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:34561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=39671
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23645
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:03.647646 23645 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:03.647974 23645 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:03.654449 23650 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.654793 23645 server_base.cc:1034] running on GCE node
W20260809 02:17:03.655242 23651 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.655856 23653 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.656127 23645 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:03.656425 23645 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:03.657586 23645 hybrid_clock.cc:648] HybridClock initialized: now 1786241823657572 us; error 30 us; skew 500 ppm
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:03.660936 23645 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:03.661530 23645 webserver.cc:460] Webserver started at http://127.9.40.66:39671/ using document root <none> and password file <none>
I20260809 02:17:03.661798 23645 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:03.661887 23645 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:03.675168 23645 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:17:03.676882 23663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.677137 23645 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260809 02:17:03.677238 23645 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "f211f0221d7e4396b6287dd236587050"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "d63c5d24eabfe1d671dc4511fa416162"
server_key_iv: "ccf79b776226f683e99ca8df09802c52"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:03.677737 23645 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:03.707441 23645 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:03.710227 23645 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:03.710413 23645 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:03.710705 23645 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:03.711135 23645 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:03.711196 23645 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.711246 23645 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:03.711267 23645 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.719718 23645 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:34561
I20260809 02:17:03.719790 23786 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:34561 every 8 connection(s)
I20260809 02:17:03.720413 23645 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:03.728976 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23645
I20260809 02:17:03.729305 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.9.40.84:58325/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:37631
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:43747
--webserver_port=35033
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37155
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:03.733178 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.721712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56361 (local address 127.9.40.126:43747)
0809 02:17:03.721757 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:03.721760 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:03.721771 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:03.722518 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:03.722520 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:03.722535 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:03.722591 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:03.723414 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.724239 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.725970 (+ 1731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.726174 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.726239 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.729119 (+ 2880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:03.729121 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:03.729121 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:03.729132 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:03.731121 (+ 1989us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.731731 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.731732 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.731733 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.731810 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.732167 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.732168 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.732168 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.732334 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:03.732414 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:03.732897 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 02:17:03.733013 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:17:03.733769 23789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.721370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.66:56361)
0809 02:17:03.721612 (+ 242us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:03.721665 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:03.722380 (+ 715us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:03.722674 (+ 294us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:03.722679 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:03.722960 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:03.723282 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.723289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.724371 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.724373 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.725866 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.725870 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.726296 (+ 426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.726298 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.726455 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.727217 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:03.727229 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:03.728896 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:03.731236 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.731245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.731248 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.731630 (+ 382us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.731898 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.731899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.731900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.732079 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.732433 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:03.732442 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:03.732644 (+ 202us) client_negotiation.cc:769] Sending connection context
0809 02:17:03.733296 (+ 652us) client_negotiation.cc:240] Negotiation successful
0809 02:17:03.733491 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":96,"threads_started":1}
I20260809 02:17:03.734469 23787 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:03.734596 23787 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.734836 23787 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
W20260809 02:17:03.735600 23787 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43747 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:17:03.735697 23787 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.735771 23787 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:03.736166 23787 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.736249 23787 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
W20260809 02:17:03.736626 23787 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:17:03.833178 23790 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:03.833411 23790 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:03.833452 23790 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:03.834664 23790 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:03.834736 23790 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:03.834784 23790 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:03.836341 23790 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37155
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.9.40.84:58325/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:37631
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=35033
--tserver_master_addrs=127.9.40.126:43747
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23790
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:03.836602 23790 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:03.836892 23790 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:03.841892 23799 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.842218 23796 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:03.842566 23797 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:03.842927 23790 server_base.cc:1034] running on GCE node
I20260809 02:17:03.843189 23790 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:03.843468 23790 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:03.844643 23790 hybrid_clock.cc:648] HybridClock initialized: now 1786241823844634 us; error 33 us; skew 500 ppm
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:03.848026 23790 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:03.848562 23790 webserver.cc:460] Webserver started at http://127.9.40.67:35033/ using document root <none> and password file <none>
I20260809 02:17:03.848901 23790 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:03.849002 23790 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:03.878285 23790 fs_manager.cc:709] Time spent opening directory manager: real 0.009s user 0.001s sys 0.000s
I20260809 02:17:03.879601 23808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.879802 23790 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:03.879894 23790 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "e681f0baff1741c3b23dcaf19a25a251"
format_stamp: "Formatted at 2026-08-09 02:17:02 on dist-test-slave-qg60"
server_key: "ffe4e2fd49669fccd78335d5cedbd45d"
server_key_iv: "ee3b37e88b2c395081ba613994565b6e"
server_key_version: "kuduclusterkey@0"
I20260809 02:17:03.880432 23790 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:03.901463 23790 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:03.904273 23790 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:03.904479 23790 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:03.904838 23790 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:03.905262 23790 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:03.905324 23790 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.905372 23790 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:03.905402 23790 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:03.914306 23790 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:37631
I20260809 02:17:03.914746 23790 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:03.915442 23928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:37631 every 8 connection(s)
I20260809 02:17:03.918912 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 23790
Aug 09 02:17:03 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241823, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:03.926901 23644 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.915925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:57523 (local address 127.9.40.126:43747)
0809 02:17:03.915989 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:03.915991 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:03.916004 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:17:03.916474 (+ 470us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:03.916475 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:03.916487 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:03.916566 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:03.917226 (+ 660us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.917997 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.919399 (+ 1402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.919756 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.919830 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.922476 (+ 2646us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:03.922477 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:03.922478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:03.922487 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:03.924633 (+ 2146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.925219 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.925221 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.925221 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.925731 (+ 510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.926078 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.926079 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.926080 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.926247 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:03.926324 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:03.926698 (+ 374us) server_negotiation.cc:299] Negotiation successful
0809 02:17:03.926785 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:17:03.927055 23931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.915610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43747 (local address 127.9.40.67:57523)
0809 02:17:03.915796 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:03.915844 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:03.916380 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:03.916623 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:03.916627 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:03.916900 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:03.917139 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.917145 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.918096 (+ 951us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.918096 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.919313 (+ 1217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:03.919316 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.919885 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.919886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:03.920029 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.920674 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:03.920685 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:03.922364 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:03.924768 (+ 2404us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.924777 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.924779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.925111 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.925822 (+ 711us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:03.925823 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:03.925824 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:03.925975 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:03.926336 (+ 361us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:03.926343 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:03.926517 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:17:03.926639 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:17:03.926817 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":101,"threads_started":1}
I20260809 02:17:03.927948 23929 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43747
I20260809 02:17:03.928084 23929 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.928298 23929 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:03.929239 23933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:03.919618 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51044 (local address 127.9.40.126:43747)
0809 02:17:03.919798 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:03.919800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:03.919808 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:17:03.919829 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:03.919830 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:03.919841 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:03.919904 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:03.920888 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.921618 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.922622 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:03.922907 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:03.922976 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:03.924785 (+ 1809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:03.924786 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:03.924787 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:03.924794 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:03.926600 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.927092 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.927093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.927094 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.927313 (+ 219us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:03.927572 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:03.927573 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:03.927573 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:03.927717 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:03.927897 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:03.929002 (+ 1105us) server_negotiation.cc:299] Negotiation successful
0809 02:17:03.929104 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
W20260809 02:17:03.929558 23929 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43747 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260809 02:17:03.929641 23929 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.929730 23929 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:03.930251 23929 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:03.930335 23929 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
W20260809 02:17:03.931316 23929 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 09 02:17:04 dist-test-slave-qg60 krb5kdc[22341](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786241824, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:04.550375 23641 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:04.550550 23641 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:04.551034 23427 ts_manager.cc:194] Registered new tserver with Master: b82c67a843044dc095347b7f43c8b180 (127.9.40.65:41319)
I20260809 02:17:04.552099 23427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:36685
I20260809 02:17:04.736828 23787 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:04.736999 23787 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:04.737350 23427 ts_manager.cc:194] Registered new tserver with Master: f211f0221d7e4396b6287dd236587050 (127.9.40.66:34561)
I20260809 02:17:04.737807 23427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:56361
Aug 09 02:17:04 dist-test-slave-qg60 krb5kdc[22341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241824, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for HTTP/127.9.40.84@KRBTEST.COM
I20260809 02:17:04.931520 23929 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:04.931699 23929 heartbeater.cc:511] Master 127.9.40.126:43747 requested a full tablet report, sending...
I20260809 02:17:04.932113 23427 ts_manager.cc:194] Registered new tserver with Master: e681f0baff1741c3b23dcaf19a25a251 (127.9.40.67:37631)
I20260809 02:17:04.932668 23427 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:57523
I20260809 02:17:04.940413 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:04.950371 23952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.941649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51048 (local address 127.9.40.126:43747)
0809 02:17:04.941908 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:04.941910 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:04.942091 (+ 181us) server_negotiation.cc:407] Connection header received
0809 02:17:04.942118 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:04.942119 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:04.942131 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:04.942188 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:04.942917 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.943798 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.944919 (+ 1121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.945249 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.945310 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.946907 (+ 1597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:04.946909 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:04.946909 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:04.946917 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:04.948751 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:04.949190 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:04.949192 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:04.949192 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:04.949257 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:04.949446 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:04.949447 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:04.949447 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:04.949584 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:04.949673 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:04.950171 (+ 498us) server_negotiation.cc:299] Negotiation successful
0809 02:17:04.950262 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":76,"threads_started":1}
I20260809 02:17:04.952935 23427 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:04.955701 23427 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:04.976626 23964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.961260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41319 (local address 127.0.0.1:42254)
0809 02:17:04.962741 (+ 1481us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:04.962782 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:04.962872 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:04.963405 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:04.963408 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:04.963410 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:04.963479 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.963481 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.970927 (+ 7446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.970928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.972251 (+ 1323us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.972255 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.976277 (+ 4022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.976278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.976372 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.976386 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:17:04.976457 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:17:04.976517 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1447,"thread_start_us":79,"threads_started":1}
I20260809 02:17:04.977152 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.961812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42254 (local address 127.9.40.65:41319)
0809 02:17:04.963183 (+ 1371us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:04.963187 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:04.963196 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:04.963230 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:04.963232 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:04.963247 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:04.963314 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:04.963915 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.970812 (+ 6897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.975314 (+ 4502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.976110 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.976184 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.976236 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:04.977019 (+ 783us) server_negotiation.cc:299] Negotiation successful
0809 02:17:04.977073 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1339,"thread_start_us":54,"threads_started":1}
I20260809 02:17:04.977670 23965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.962427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59878 (local address 127.9.40.67:37631)
0809 02:17:04.962997 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:04.963002 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:04.967001 (+ 3999us) server_negotiation.cc:407] Connection header received
0809 02:17:04.967040 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:04.967042 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:04.967058 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:04.967123 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:04.967406 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.969882 (+ 2476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.974297 (+ 4415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.975126 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.975201 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.975268 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:04.977537 (+ 2269us) server_negotiation.cc:299] Negotiation successful
0809 02:17:04.977590 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":539,"thread_start_us":58,"threads_started":1}
I20260809 02:17:04.977828 23959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.962133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:37631 (local address 127.0.0.1:59878)
0809 02:17:04.966886 (+ 4753us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:04.966896 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:04.966942 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:04.967289 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:04.967290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:04.967292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:04.967332 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.967333 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.972354 (+ 5021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.972355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.974190 (+ 1835us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.974192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.976970 (+ 2778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.976971 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.977032 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.977037 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:17:04.977092 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:17:04.977131 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4737,"thread_start_us":67,"threads_started":1}
I20260809 02:17:04.977993 23570 tablet_service.cc:1467] Processing CreateTablet for tablet fb525c232f3541299a30453fd46c3fe1 (DEFAULT_TABLE table=test-table [id=2d4c5047a0be41d995f7fdffcf07d772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:04.978392 23570 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb525c232f3541299a30453fd46c3fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:04.978649 23858 tablet_service.cc:1467] Processing CreateTablet for tablet fb525c232f3541299a30453fd46c3fe1 (DEFAULT_TABLE table=test-table [id=2d4c5047a0be41d995f7fdffcf07d772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:04.978976 23858 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb525c232f3541299a30453fd46c3fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:04.982931 23966 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Bootstrap starting.
I20260809 02:17:04.983683 23966 tablet_bootstrap.cc:654] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:04.984010 23966 log.cc:824] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:04.984345 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.961489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:34561 (local address 127.0.0.1:48700)
0809 02:17:04.963044 (+ 1555us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:04.963052 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:04.963088 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:04.963806 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:04.963807 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:04.963809 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:04.963842 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.963843 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.965785 (+ 1942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.965786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.973332 (+ 7546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:04.973336 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.984006 (+ 10670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.984008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:04.984130 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.984136 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:17:04.984194 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 02:17:04.984249 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1541,"thread_start_us":81,"threads_started":1}
I20260809 02:17:04.984369 23967 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Bootstrap starting.
I20260809 02:17:04.984699 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:04.961940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48700 (local address 127.9.40.66:34561)
0809 02:17:04.963605 (+ 1665us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:04.963609 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:04.963618 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:04.963649 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:04.963651 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:04.963665 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:04.963730 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:04.964582 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.965684 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.978855 (+ 13171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:04.983800 (+ 4945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:04.983883 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:04.983942 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:04.984558 (+ 616us) server_negotiation.cc:299] Negotiation successful
0809 02:17:04.984613 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1631,"thread_start_us":50,"threads_started":1}
I20260809 02:17:04.985513 23704 tablet_service.cc:1467] Processing CreateTablet for tablet fb525c232f3541299a30453fd46c3fe1 (DEFAULT_TABLE table=test-table [id=2d4c5047a0be41d995f7fdffcf07d772]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:04.985898 23704 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fb525c232f3541299a30453fd46c3fe1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:04.990135 23967 tablet_bootstrap.cc:654] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:04.990478 23967 log.cc:824] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:04.990840 23969 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Bootstrap starting.
I20260809 02:17:04.991452 23969 tablet_bootstrap.cc:654] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:04.991760 23969 log.cc:824] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:04.992586 23967 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: No bootstrap required, opened a new log
I20260809 02:17:04.992704 23967 ts_tablet_manager.cc:1397] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.000s
I20260809 02:17:04.995709 23967 raft_consensus.cc:348] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:04.995927 23967 raft_consensus.cc:374] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:04.995962 23967 raft_consensus.cc:729] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b82c67a843044dc095347b7f43c8b180, State: Initialized, Role: FOLLOWER
I20260809 02:17:04.996119 23967 consensus_queue.cc:260] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:04.996405 23967 ts_tablet_manager.cc:1428] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260809 02:17:04.996531 23641 heartbeater.cc:503] Master 127.9.40.126:43747 was elected leader, sending a full tablet report...
I20260809 02:17:04.998579 23969 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: No bootstrap required, opened a new log
I20260809 02:17:04.998663 23969 ts_tablet_manager.cc:1397] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260809 02:17:04.999236 23966 tablet_bootstrap.cc:492] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: No bootstrap required, opened a new log
I20260809 02:17:04.999318 23966 ts_tablet_manager.cc:1397] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.000s
I20260809 02:17:05.001852 23966 raft_consensus.cc:348] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.002056 23966 raft_consensus.cc:374] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:05.002097 23966 raft_consensus.cc:729] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e681f0baff1741c3b23dcaf19a25a251, State: Initialized, Role: FOLLOWER
I20260809 02:17:05.002523 23966 consensus_queue.cc:260] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.002496 23969 raft_consensus.cc:348] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.002777 23969 raft_consensus.cc:374] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:05.002852 23969 raft_consensus.cc:729] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f211f0221d7e4396b6287dd236587050, State: Initialized, Role: FOLLOWER
I20260809 02:17:05.002876 23966 ts_tablet_manager.cc:1428] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260809 02:17:05.002991 23929 heartbeater.cc:503] Master 127.9.40.126:43747 was elected leader, sending a full tablet report...
I20260809 02:17:05.003084 23969 consensus_queue.cc:260] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.003433 23969 ts_tablet_manager.cc:1428] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260809 02:17:05.003665 23787 heartbeater.cc:503] Master 127.9.40.126:43747 was elected leader, sending a full tablet report...
W20260809 02:17:05.017586 23642 tablet.cc:2369] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:05.059687 23974 raft_consensus.cc:482] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:05.059856 23974 raft_consensus.cc:504] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.060871 23974 leader_election.cc:288] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b82c67a843044dc095347b7f43c8b180 (127.9.40.65:41319), e681f0baff1741c3b23dcaf19a25a251 (127.9.40.67:37631)
I20260809 02:17:05.064889 23961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.060808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:46763 (local address 127.9.40.65:41319)
0809 02:17:05.060858 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:05.060860 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:05.061092 (+ 232us) server_negotiation.cc:407] Connection header received
0809 02:17:05.061208 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:05.061210 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:05.061225 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:05.061256 (+ 31us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:05.061523 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.062242 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.063852 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.064417 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.064467 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:05.064479 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:05.064752 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 02:17:05.064804 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:17:05.064934 23965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.060794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52279 (local address 127.9.40.67:37631)
0809 02:17:05.060845 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:05.060847 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:05.061081 (+ 234us) server_negotiation.cc:407] Connection header received
0809 02:17:05.061195 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:05.061196 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:05.061208 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:05.061241 (+ 33us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:05.061495 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.062285 (+ 790us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.063863 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.064438 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.064490 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:05.064502 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:05.064807 (+ 305us) server_negotiation.cc:299] Negotiation successful
0809 02:17:05.064852 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:17:05.065186 23975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.060513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41319 (local address 127.9.40.66:46763)
0809 02:17:05.061006 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:05.061017 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:05.061125 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:05.061332 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:05.061333 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:05.061335 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:05.061429 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:05.061431 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.062356 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.062357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:05.063739 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:05.063742 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.064563 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.064564 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:05.064635 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:05.064640 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:17:05.065030 (+ 390us) client_negotiation.cc:240] Negotiation successful
0809 02:17:05.065086 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":476,"mutex_wait_us":91,"thread_start_us":143,"threads_started":1}
I20260809 02:17:05.065531 23593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb525c232f3541299a30453fd46c3fe1" candidate_uuid: "f211f0221d7e4396b6287dd236587050" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b82c67a843044dc095347b7f43c8b180" is_pre_election: true
I20260809 02:17:05.065613 23976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.060509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:37631 (local address 127.9.40.66:52279)
0809 02:17:05.060933 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:05.061015 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:05.061114 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:05.061326 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:05.061327 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:05.061329 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:05.061416 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:05.061418 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.062394 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.062395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:05.063745 (+ 1350us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:05.063749 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.064567 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.064568 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:05.064672 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:05.064676 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:05.065467 (+ 791us) client_negotiation.cc:240] Negotiation successful
0809 02:17:05.065518 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"mutex_wait_us":3,"thread_start_us":138,"threads_started":1}
I20260809 02:17:05.065711 23593 raft_consensus.cc:2454] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f211f0221d7e4396b6287dd236587050 in term 0.
I20260809 02:17:05.065948 23666 leader_election.cc:302] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b82c67a843044dc095347b7f43c8b180, f211f0221d7e4396b6287dd236587050; no voters:
I20260809 02:17:05.066098 23974 raft_consensus.cc:2781] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:05.066144 23974 raft_consensus.cc:482] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:05.066165 23974 raft_consensus.cc:3037] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:05.066147 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb525c232f3541299a30453fd46c3fe1" candidate_uuid: "f211f0221d7e4396b6287dd236587050" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e681f0baff1741c3b23dcaf19a25a251" is_pre_election: true
I20260809 02:17:05.066298 23881 raft_consensus.cc:2454] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f211f0221d7e4396b6287dd236587050 in term 0.
I20260809 02:17:05.067150 23974 raft_consensus.cc:504] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.067530 23593 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb525c232f3541299a30453fd46c3fe1" candidate_uuid: "f211f0221d7e4396b6287dd236587050" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b82c67a843044dc095347b7f43c8b180"
I20260809 02:17:05.067543 23881 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fb525c232f3541299a30453fd46c3fe1" candidate_uuid: "f211f0221d7e4396b6287dd236587050" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e681f0baff1741c3b23dcaf19a25a251"
I20260809 02:17:05.067615 23593 raft_consensus.cc:3037] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:05.067619 23881 raft_consensus.cc:3037] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:05.068522 23593 raft_consensus.cc:2454] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f211f0221d7e4396b6287dd236587050 in term 1.
I20260809 02:17:05.068600 23881 raft_consensus.cc:2454] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f211f0221d7e4396b6287dd236587050 in term 1.
I20260809 02:17:05.068811 23665 leader_election.cc:302] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: e681f0baff1741c3b23dcaf19a25a251, f211f0221d7e4396b6287dd236587050; no voters:
I20260809 02:17:05.069060 23974 leader_election.cc:288] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [CANDIDATE]: Term 1 election: Requested vote from peers b82c67a843044dc095347b7f43c8b180 (127.9.40.65:41319), e681f0baff1741c3b23dcaf19a25a251 (127.9.40.67:37631)
I20260809 02:17:05.069083 23977 raft_consensus.cc:2781] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:05.069234 23977 raft_consensus.cc:686] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 1 LEADER]: Becoming Leader. State: Replica: f211f0221d7e4396b6287dd236587050, State: Running, Role: LEADER
I20260809 02:17:05.069347 23977 consensus_queue.cc:237] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } }
I20260809 02:17:05.070380 23427 catalog_manager.cc:5495] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 reported cstate change: term changed from 0 to 1, leader changed from <none> to f211f0221d7e4396b6287dd236587050 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "f211f0221d7e4396b6287dd236587050" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f211f0221d7e4396b6287dd236587050" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 34561 } health_report { overall_health: HEALTHY } } }
I20260809 02:17:05.097507 23962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.094108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48708 (local address 127.9.40.66:34561)
0809 02:17:05.094192 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:05.094194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:05.094203 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:05.094251 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:05.094252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:05.094272 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:05.094378 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:05.094563 (+ 185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.095285 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.096405 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:05.096689 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:05.096781 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:05.096991 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:05.097149 (+ 158us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:05.097229 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:05.097402 (+ 173us) server_negotiation.cc:299] Negotiation successful
0809 02:17:05.097421 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260809 02:17:05.100019 23593 raft_consensus.cc:1261] T fb525c232f3541299a30453fd46c3fe1 P b82c67a843044dc095347b7f43c8b180 [term 1 FOLLOWER]: Refusing update from remote peer f211f0221d7e4396b6287dd236587050: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:05.100224 23881 raft_consensus.cc:1261] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 1 FOLLOWER]: Refusing update from remote peer f211f0221d7e4396b6287dd236587050: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:05.100668 23974 consensus_queue.cc:1035] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b82c67a843044dc095347b7f43c8b180" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 41319 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:05.100875 23974 consensus_queue.cc:1035] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e681f0baff1741c3b23dcaf19a25a251" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37631 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:05.107503 23981 mvcc.cc:205] Tried to move back new op lower bound from 7316446515605786624 to 7316446515484778496. Current Snapshot: MvccSnapshot[applied={T|T < 7316446515605786624}]
I20260809 02:17:05.110020 23426 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:05.110146 23426 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51048:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:05.110939 23426 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3f6f9c2e2ce04316a9314ed7c3226eb3: Sending DeleteTablet for 3 replicas of tablet fb525c232f3541299a30453fd46c3fe1
I20260809 02:17:05.111333 23858 tablet_service.cc:1514] Processing DeleteTablet for tablet fb525c232f3541299a30453fd46c3fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:59878
I20260809 02:17:05.111438 23570 tablet_service.cc:1514] Processing DeleteTablet for tablet fb525c232f3541299a30453fd46c3fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42254
I20260809 02:17:05.111656 23569 tablet_service.cc:1514] Processing DeleteTablet for tablet fb525c232f3541299a30453fd46c3fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:42254
I20260809 02:17:05.111860 23704 tablet_service.cc:1514] Processing DeleteTablet for tablet fb525c232f3541299a30453fd46c3fe1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:05 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:48700
I20260809 02:17:05.111939 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23471
I20260809 02:17:05.115576 23990 tablet_replica.cc:330] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: stopping tablet replica
I20260809 02:17:05.115762 23990 raft_consensus.cc:2229] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:05.115863 23990 raft_consensus.cc:2258] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:05.118283 23992 tablet_replica.cc:330] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: stopping tablet replica
I20260809 02:17:05.118431 23992 raft_consensus.cc:2229] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:05.118600 23992 raft_consensus.cc:2258] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:05.119168 23992 ts_tablet_manager.cc:1905] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:05.121730 23990 ts_tablet_manager.cc:1905] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:05.121958 23992 ts_tablet_manager.cc:1918] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:05.122121 23992 log.cc:1196] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/fb525c232f3541299a30453fd46c3fe1
I20260809 02:17:05.122457 23992 ts_tablet_manager.cc:1939] T fb525c232f3541299a30453fd46c3fe1 P f211f0221d7e4396b6287dd236587050: Deleting consensus metadata
I20260809 02:17:05.123854 23990 ts_tablet_manager.cc:1918] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:05.123950 23990 log.cc:1196] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/fb525c232f3541299a30453fd46c3fe1
I20260809 02:17:05.124272 23990 ts_tablet_manager.cc:1939] T fb525c232f3541299a30453fd46c3fe1 P e681f0baff1741c3b23dcaf19a25a251: Deleting consensus metadata
I20260809 02:17:05.124607 23413 catalog_manager.cc:4841] TS f211f0221d7e4396b6287dd236587050 (127.9.40.66:34561): tablet fb525c232f3541299a30453fd46c3fe1 (table test-table [id=2d4c5047a0be41d995f7fdffcf07d772]) successfully deleted
W20260809 02:17:05.124871 23666 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:41319 (error 108)
I20260809 02:17:05.124898 23410 catalog_manager.cc:4841] TS e681f0baff1741c3b23dcaf19a25a251 (127.9.40.67:37631): tablet fb525c232f3541299a30453fd46c3fe1 (table test-table [id=2d4c5047a0be41d995f7fdffcf07d772]) successfully deleted
W20260809 02:17:05.125344 23411 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:41319 (error 108)
I20260809 02:17:05.126274 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.126120 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41319 (local address 127.0.0.1:42270)
0809 02:17:05.126195 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:05.126237 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 02:17:05.126408 23411 catalog_manager.cc:4584] TS b82c67a843044dc095347b7f43c8b180 (127.9.40.65:41319): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fb525c232f3541299a30453fd46c3fe1: Network error: Client connection negotiation failed: client connection to 127.9.40.65:41319: connect: Connection refused (error 111)
I20260809 02:17:05.126637 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23645
I20260809 02:17:05.145485 23960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:05.145302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:41319 (local address 127.0.0.1:42274)
0809 02:17:05.145376 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:05.145429 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:41319: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:17:05.150954 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23790
I20260809 02:17:05.161849 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 23392
I20260809 02:17:05.171039 9377 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260809 02:17:05.515940 9377 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260809 02:17:05.516043 9377 mini_ranger.cc:66] Stopping Ranger...
I20260809 02:17:05.861302 9377 mini_ranger.cc:68] Stopped Ranger
2026-08-09 02:17:05.861 UTC [22373] LOG: received smart shutdown request
2026-08-09 02:17:05.866 UTC [22373] LOG: background worker "logical replication launcher" (PID 22383) exited with exit code 1
2026-08-09 02:17:05.867 UTC [22377] LOG: shutting down
2026-08-09 02:17:06.057 UTC [22373] LOG: database system is shut down
2026-08-09T02:17:06Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (50044 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:17:06 dist-test-slave-qg60 krb5kdc[24003](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:17:06 dist-test-slave-qg60 krb5kdc[24003](info): set up 2 sockets
Aug 09 02:17:06 dist-test-slave-qg60 krb5kdc[24003](info): commencing operation
krb5kdc: starting...
W20260809 02:17:07.289592 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.094s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:17:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:17:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:17:07.424053 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46209
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:07.508525 24018 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:07.508769 24018 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:07.508795 24018 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:07.509909 24018 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:07.509946 24018 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:07.509960 24018 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:07.509974 24018 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:07.509985 24018 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:07.511418 24018 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24018
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:07.511627 24018 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:07.511802 24018 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:07.516780 24027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:07.516764 24024 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:07.516765 24025 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:07.516906 24018 server_base.cc:1034] running on GCE node
I20260809 02:17:07.517228 24018 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:07.517521 24018 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:07.518664 24018 hybrid_clock.cc:648] HybridClock initialized: now 1786241827518653 us; error 28 us; skew 500 ppm
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:07.524371 24018 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:07.524959 24018 webserver.cc:460] Webserver started at http://127.9.40.126:42675/ using document root <none> and password file <none>
I20260809 02:17:07.525229 24018 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:07.525305 24018 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:07.525461 24018 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:07.526572 24018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "6619c5e9977f4d7d822775f20bd05475"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "9bbfaddd584c0ded4ef1e8e717b362a0"
server_key_iv: "3bb6d5460112b244adcba9061e6cf8f6"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.526954 24018 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "6619c5e9977f4d7d822775f20bd05475"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "9bbfaddd584c0ded4ef1e8e717b362a0"
server_key_iv: "3bb6d5460112b244adcba9061e6cf8f6"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.528335 24018 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:17:07.529201 24034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:07.529409 24018 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:07.529497 24018 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "6619c5e9977f4d7d822775f20bd05475"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "9bbfaddd584c0ded4ef1e8e717b362a0"
server_key_iv: "3bb6d5460112b244adcba9061e6cf8f6"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.529554 24018 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:07.556277 24018 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:07.558995 24018 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:07.559172 24018 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:07.563470 24018 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46209
I20260809 02:17:07.563550 24089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46209 every 8 connection(s)
I20260809 02:17:07.563949 24018 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:07.565095 24090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:07.566861 24090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:17:07.570106 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24018
I20260809 02:17:07.570232 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:17:07.570459 9377 external_mini_cluster.cc:1428] Setting key b19587f7726627c764dbc2cd3d99488a
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:07.578750 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.567548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46209 (local address 127.0.0.1:53782)
0809 02:17:07.567713 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.567764 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.568329 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.568851 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.568854 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.569333 (+ 479us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.569619 (+ 286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.569626 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.570804 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.570805 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.572056 (+ 1251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.572060 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.572577 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.572578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.572645 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.573256 (+ 611us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.573265 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.574998 (+ 1733us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.577213 (+ 2215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.577220 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.577222 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.577420 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.577682 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.577682 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.577683 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.577774 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.578141 (+ 367us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.578146 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.578208 (+ 62us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.578321 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.578561 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"mutex_wait_us":95,"thread_start_us":67,"threads_started":1}
I20260809 02:17:07.578751 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.567619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53782 (local address 127.9.40.126:46209)
0809 02:17:07.567805 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.567809 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.567857 (+ 48us) server_negotiation.cc:407] Connection header received
0809 02:17:07.568666 (+ 809us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.568668 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.568690 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.568826 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.569726 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.570706 (+ 980us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.572156 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.572470 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.572592 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.575243 (+ 2651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.575252 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.575252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.575260 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.577109 (+ 1849us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.577527 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.577530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.577530 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.577611 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.577841 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.577842 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.577843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.578067 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.578163 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.578346 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.578495 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":121,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
I20260809 02:17:07.579859 24090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:17:07.580322 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.580148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54650)
0809 02:17:07.580227 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.580286 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
W20260809 02:17:07.580446 24036 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
I20260809 02:17:07.580891 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.580791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54664)
0809 02:17:07.580847 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.580873 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260809 02:17:07.581019 24090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44647: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260809 02:17:07.599959 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:44647
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:07.630326 24090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } attempt: 1
I20260809 02:17:07.630898 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.630740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54668)
0809 02:17:07.630812 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.630863 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:17:07.631306 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.631200 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54678)
0809 02:17:07.631252 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.631284 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:17:07.631472 24090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44647: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
W20260809 02:17:07.684839 24098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:07.685065 24098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:07.685089 24098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:07.686264 24098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:07.686311 24098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:07.686326 24098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:07.686339 24098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:07.686396 24098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:07.687916 24098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:44647
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:07.688181 24098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:07.688421 24098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:07.693507 24107 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:07.693614 24105 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:07.693637 24098 server_base.cc:1034] running on GCE node
W20260809 02:17:07.693753 24104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:07.694068 24098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:07.694377 24098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:07.695576 24098 hybrid_clock.cc:648] HybridClock initialized: now 1786241827695565 us; error 33 us; skew 500 ppm
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:07.698271 24098 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260809 02:17:07.698671 24098 webserver.cc:460] Webserver started at http://127.9.40.125:39329/ using document root <none> and password file <none>
I20260809 02:17:07.698877 24098 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:07.698927 24098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:07.699030 24098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:07.699630 24090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } attempt: 2
I20260809 02:17:07.700026 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.699892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54680)
0809 02:17:07.699958 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.699995 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:17:07.700284 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data/instance:
uuid: "d5f6f91fe82a470c947429ffd53fc649"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "2f47e5ef8bef17eb136262a3f74e5fb5"
server_key_iv: "d6720693ce193c12e847b1e3829c08e3"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.700413 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.700317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54686)
0809 02:17:07.700364 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.700395 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:17:07.700537 24090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44647: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44647: connect: Connection refused (error 111)
I20260809 02:17:07.700699 24098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal/instance:
uuid: "d5f6f91fe82a470c947429ffd53fc649"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "2f47e5ef8bef17eb136262a3f74e5fb5"
server_key_iv: "d6720693ce193c12e847b1e3829c08e3"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.702113 24098 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:07.703064 24114 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:07.703264 24098 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:07.703363 24098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "d5f6f91fe82a470c947429ffd53fc649"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "2f47e5ef8bef17eb136262a3f74e5fb5"
server_key_iv: "d6720693ce193c12e847b1e3829c08e3"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.703467 24098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:07.717144 24098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:07.718993 24098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:07.719172 24098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:07.723695 24098 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:44647
I20260809 02:17:07.723749 24170 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:44647 every 8 connection(s)
I20260809 02:17:07.724144 24098 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
I20260809 02:17:07.725132 24171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:07.725688 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24098
I20260809 02:17:07.725816 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-1/wal/instance
I20260809 02:17:07.726008 9377 external_mini_cluster.cc:1428] Setting key 056dcfc5a1c53dc139484889dd64759f
I20260809 02:17:07.727733 24171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
I20260809 02:17:07.738885 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.728669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53798 (local address 127.9.40.126:46209)
0809 02:17:07.728775 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.728778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.729088 (+ 310us) server_negotiation.cc:407] Connection header received
0809 02:17:07.729877 (+ 789us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.729878 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.729891 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.729934 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.730867 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.731497 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.733206 (+ 1709us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.733438 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.733498 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.735984 (+ 2486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.735985 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.735985 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.735993 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.737380 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.737839 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.737840 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.737840 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.737890 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.738132 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.738132 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.738133 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.738239 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.738293 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.738630 (+ 337us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.738733 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:17:07.738957 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.728637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46209 (local address 127.0.0.1:53798)
0809 02:17:07.728872 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.728944 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.729749 (+ 805us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.730010 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.730014 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.730446 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.730759 (+ 313us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.730766 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.731618 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.731619 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.733101 (+ 1482us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.733105 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.733525 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.733527 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.733681 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.734241 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.734254 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.735890 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.737460 (+ 1570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.737467 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.737469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.737716 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.737961 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.737962 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.737962 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.738069 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.738324 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.738331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.738493 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.738582 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.738762 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
I20260809 02:17:07.739570 24171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:17:07.748087 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.739871 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54694)
0809 02:17:07.739923 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.739935 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.739975 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.740283 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.740284 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.740399 (+ 115us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.740449 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.740451 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.741412 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.741414 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.742103 (+ 689us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.742105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.742770 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.742770 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.742829 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.743317 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.743324 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.744841 (+ 1517us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.746927 (+ 2086us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.746928 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.746928 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.747101 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.747380 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.747381 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.747381 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.747452 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.747826 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.747826 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.747871 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.747939 (+ 68us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.748017 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:17:07.748265 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.739932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54694 (local address 127.9.40.125:44647)
0809 02:17:07.740115 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.740121 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.740149 (+ 28us) server_negotiation.cc:407] Connection header received
0809 02:17:07.740175 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.740177 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.740192 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.740275 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.740523 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.741322 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.742198 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.742685 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.742763 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.744927 (+ 2164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.744937 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.744937 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.744945 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.746810 (+ 1865us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.747208 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.747212 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.747212 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.747280 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.747534 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.747535 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.747535 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.747743 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.747839 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.747990 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.748094 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":6,"server-negotiator.queue_time_us":142,"thread_start_us":93,"threads_started":1}
I20260809 02:17:07.749089 24171 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:17:07.749501 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.749341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41776)
0809 02:17:07.749393 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.749482 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260809 02:17:07.749640 24117 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
I20260809 02:17:07.750068 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.749967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41782)
0809 02:17:07.750022 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.750046 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:17:07.750185 24171 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46115: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260809 02:17:07.755856 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46115
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33173
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:07.791698 24090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } attempt: 3
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:17:07.799388 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.792154 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54710)
0809 02:17:07.792225 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.792239 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.792298 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.792447 (+ 149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.792447 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.792632 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.792701 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.792702 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.793461 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.793462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.793979 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.793981 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.794373 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.794374 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.794488 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.795005 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.795010 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.796456 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.798332 (+ 1876us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.798333 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.798334 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.798515 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.798732 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.798733 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.798733 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.798788 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.799060 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.799061 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.799113 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.799173 (+ 60us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.799259 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:17:07.799397 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.792233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54710 (local address 127.9.40.125:44647)
0809 02:17:07.792313 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.792316 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.792326 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:17:07.792358 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.792359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.792376 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.792441 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.792836 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.793359 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.794072 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.794278 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.794371 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.796557 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.796559 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.796559 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.796569 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.798218 (+ 1649us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.798595 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.798596 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.798596 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.798661 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.798849 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.798849 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.798850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.798988 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.799056 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.799217 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.799302 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:17:07.799418 24171 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } attempt: 1
I20260809 02:17:07.799896 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.799753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41798)
0809 02:17:07.799823 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.799865 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:17:07.800266 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.800172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41800)
0809 02:17:07.800210 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.800235 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 02:17:07.800241 24090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
W20260809 02:17:07.800351 24171 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46115: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
I20260809 02:17:07.800592 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.800481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41814)
0809 02:17:07.800534 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.800566 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:17:07.800956 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.800880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41830)
0809 02:17:07.800909 (+ 29us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.800936 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260809 02:17:07.801046 24090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46115: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
I20260809 02:17:07.832195 24090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } attempt: 1
I20260809 02:17:07.832835 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.832567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41846)
0809 02:17:07.832694 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.832790 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95}
I20260809 02:17:07.833315 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.833188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41858)
0809 02:17:07.833258 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.833291 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:17:07.833490 24090 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46115: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
W20260809 02:17:07.843060 24179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:07.843322 24179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:07.843360 24179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:07.844528 24179 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:07.844599 24179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:07.844626 24179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:07.844663 24179 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:07.844689 24179 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:07.846300 24179 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33173
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46209,127.9.40.125:44647,127.9.40.124:46115
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46115
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:07.846606 24179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:07.846858 24179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:07.851976 24184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:07.851984 24186 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:07.852171 24179 server_base.cc:1034] running on GCE node
W20260809 02:17:07.851984 24188 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:07.852443 24179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:07.852815 24179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:07.854095 24179 hybrid_clock.cc:648] HybridClock initialized: now 1786241827854036 us; error 76 us; skew 500 ppm
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:07.856824 24179 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260809 02:17:07.857306 24179 webserver.cc:460] Webserver started at http://127.9.40.124:33033/ using document root <none> and password file <none>
I20260809 02:17:07.857558 24179 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:07.857627 24179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:07.857784 24179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:07.858973 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data/instance:
uuid: "a48b9366e5844839bcdc63dccb5da611"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "493ca8654349ec784689ac99e2a958b4"
server_key_iv: "f30884f4147f17011489bbfb657167b9"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.859373 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal/instance:
uuid: "a48b9366e5844839bcdc63dccb5da611"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "493ca8654349ec784689ac99e2a958b4"
server_key_iv: "f30884f4147f17011489bbfb657167b9"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.860906 24179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 02:17:07.862004 24196 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:07.862273 24179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:07.862397 24179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "a48b9366e5844839bcdc63dccb5da611"
format_stamp: "Formatted at 2026-08-09 02:17:07 on dist-test-slave-qg60"
server_key: "493ca8654349ec784689ac99e2a958b4"
server_key_iv: "f30884f4147f17011489bbfb657167b9"
server_key_version: "encryptionkey@0"
I20260809 02:17:07.862483 24179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:07.868494 24171 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } attempt: 2
I20260809 02:17:07.869030 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.868875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41866)
0809 02:17:07.868951 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.868997 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260809 02:17:07.869334 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.869267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41878)
0809 02:17:07.869293 (+ 26us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.869317 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
W20260809 02:17:07.869458 24171 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46115: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46115: connect: Connection refused (error 111)
I20260809 02:17:07.874539 24179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:07.876523 24179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:07.876695 24179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:07.880897 24179 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:46115
I20260809 02:17:07.880993 24250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:46115 every 8 connection(s)
I20260809 02:17:07.881335 24179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
I20260809 02:17:07.881840 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24179
I20260809 02:17:07.881974 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786241669736972-9377-0/minicluster-data/master-2/wal/instance
I20260809 02:17:07.882174 9377 external_mini_cluster.cc:1428] Setting key 6316824f6963c6526ca386b3c883729e
I20260809 02:17:07.882529 24251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:07.885087 24251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:07.894084 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.885797 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53804 (local address 127.9.40.126:46209)
0809 02:17:07.885855 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.885857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.886019 (+ 162us) server_negotiation.cc:407] Connection header received
0809 02:17:07.886065 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.886066 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.886084 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.886124 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.886691 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.887304 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.888241 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.888437 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.888515 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.891083 (+ 2568us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.891084 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.891084 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.891091 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.892841 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.893278 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.893279 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.893279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.893327 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.893560 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.893561 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.893561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.893697 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.893771 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.893954 (+ 183us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.894020 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:17:07.894084 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.882708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41880 (local address 127.9.40.124:46115)
0809 02:17:07.882909 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.882916 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.882942 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:17:07.883717 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.883724 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.883771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.883885 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.884681 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.885923 (+ 1242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.886927 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.887566 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.887644 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.890604 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.890613 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.890613 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.890620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.892235 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.892648 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.892651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.892651 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.892780 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.893009 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.893009 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.893010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.893332 (+ 322us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.893485 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.893636 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.893754 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":103,"threads_started":1}
I20260809 02:17:07.894101 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.885698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46209 (local address 127.0.0.1:53804)
0809 02:17:07.885881 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.885924 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.886008 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.886178 (+ 170us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.886181 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.886519 (+ 338us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.886600 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.886602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.887396 (+ 794us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.887397 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.888127 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.888155 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.888539 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.888540 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.888628 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.889158 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.889168 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.891008 (+ 1840us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.892920 (+ 1912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.892925 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.892927 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.893204 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.893394 (+ 190us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.893395 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.893396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.893459 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.893775 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.893780 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.893827 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.893916 (+ 89us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.894012 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":83,"threads_started":1}
I20260809 02:17:07.894903 24251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:07.902684 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.895181 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44647 (local address 127.0.0.1:54720)
0809 02:17:07.895236 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.895250 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.895297 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.895434 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.895434 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.895596 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.895639 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.895642 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.896629 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.896630 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.897484 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.897486 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.897918 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.897919 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.897965 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.898376 (+ 411us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.898380 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.899819 (+ 1439us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.901653 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.901654 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.901655 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.901850 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.902079 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.902080 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.902080 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.902131 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.902415 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.902415 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.902452 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.902502 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.902579 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:17:07.902748 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.895254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54720 (local address 127.9.40.125:44647)
0809 02:17:07.895298 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.895300 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.895309 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:07.895360 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.895361 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.895372 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.895432 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.895742 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.896535 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.897574 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.897840 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.897913 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.899915 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.899916 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.899917 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.899926 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.901561 (+ 1635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.901929 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.901930 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.901930 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.902005 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.902196 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.902197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.902197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.902343 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.902422 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.902532 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.902614 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:17:07.903533 24251 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:17:07.904273 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.895082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53814 (local address 127.9.40.126:46209)
0809 02:17:07.895125 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.895127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.895310 (+ 183us) server_negotiation.cc:407] Connection header received
0809 02:17:07.895367 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.895368 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.895377 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.895432 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.896072 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.896773 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.897697 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.898031 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.898084 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.900654 (+ 2570us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.900655 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.900655 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.900662 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.902627 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.903067 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.903068 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.903068 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.903121 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.903347 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.903349 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.903349 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.903486 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.903697 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.904088 (+ 391us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.904154 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:17:07.908653 24090 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } attempt: 2
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:17:07.911095 24256 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.903811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41884)
0809 02:17:07.903858 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.903868 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.903900 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.904322 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.904323 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.904460 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.904493 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.904495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.905299 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.905300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.905772 (+ 472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.905774 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.906162 (+ 388us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.906163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.906210 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.906661 (+ 451us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.906665 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.907919 (+ 1254us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.909764 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.909765 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.909765 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.909969 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.910153 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.910154 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.910154 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.910202 (+ 48us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.910772 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.910773 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.910823 (+ 50us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.910900 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.910995 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:17:07.911429 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.904142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41884 (local address 127.9.40.124:46115)
0809 02:17:07.904205 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.904207 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.904215 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:17:07.904233 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.904234 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.904243 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.904295 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.904583 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.905211 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.905854 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.906064 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.906140 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.908025 (+ 1885us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.908026 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.908026 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.908033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.909679 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.910042 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.910042 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.910043 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.910095 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.910267 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.910268 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.910269 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.910401 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.910488 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.911274 (+ 786us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.911347 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:17:07.913242 24251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: Bootstrap starting.
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:17:07.914093 24251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:07.914170 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.905060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54736 (local address 127.9.40.125:44647)
0809 02:17:07.905124 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.905126 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.905134 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:17:07.905157 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.905158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.905166 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.905216 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.905698 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.906213 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.907127 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.907395 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.907501 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.910042 (+ 2541us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.910044 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.910044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.910051 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.911856 (+ 1805us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.913157 (+ 1301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.913159 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.913159 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.913338 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.913570 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.913571 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.913571 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.913693 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.913926 (+ 233us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.913966 (+ 40us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.914048 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:17:07.914438 24251 log.cc:824] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:07.915688 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:17:07.915745 24251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: No bootstrap required, opened a new log
I20260809 02:17:07.917291 24092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.908929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41896)
0809 02:17:07.909098 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.909110 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.909157 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.909601 (+ 444us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.909603 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.909768 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.909818 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.909820 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.911040 (+ 1220us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.911042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.911816 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.911819 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.912211 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.912212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.912267 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.912593 (+ 326us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.912598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.914053 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.916071 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.916072 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.916072 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.916314 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.916544 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.916544 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.916545 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.916610 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.916978 (+ 368us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.916979 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.917028 (+ 49us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.917079 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.917192 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149}
I20260809 02:17:07.917670 24257 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.909230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41896 (local address 127.9.40.124:46115)
0809 02:17:07.909449 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.909451 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.909461 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:17:07.909481 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.909482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.909494 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.909571 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.909914 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.910967 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.911915 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.912130 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.912181 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.914159 (+ 1978us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.914160 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.914160 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.914167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.915982 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.916400 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.916401 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.916402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.916463 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.916686 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.916687 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.916688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.916882 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.917412 (+ 530us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.917466 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.917544 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":193,"thread_start_us":128,"threads_started":1}
I20260809 02:17:07.918020 24251 raft_consensus.cc:348] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.918292 24251 raft_consensus.cc:374] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:07.918396 24251 raft_consensus.cc:729] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a48b9366e5844839bcdc63dccb5da611, State: Initialized, Role: FOLLOWER
I20260809 02:17:07.918625 24251 consensus_queue.cc:260] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.919065 24251 sys_catalog.cc:564] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:07.919853 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: Bootstrap starting.
I20260809 02:17:07.920607 24090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:07.920955 24090 log.cc:824] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: Log is configured to *not* fsync() on all Append() calls
W20260809 02:17:07.921389 24279 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:17:07.921500 24279 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:17:07.921869 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:07.922057 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:07.922309 24179 master_runner.cc:186] Error getting master registration for 127.9.40.126:46209: OK, instance_id { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" instance_seqno: 1786241827562184 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:17:07.925192 24179 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:17:07.925751 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.917588 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41912 (local address 127.9.40.124:46115)
0809 02:17:07.917624 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.917626 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.917994 (+ 368us) server_negotiation.cc:407] Connection header received
0809 02:17:07.918072 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.918073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.918083 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.918149 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.918875 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.919583 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.920825 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.921099 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.921157 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.922668 (+ 1511us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.922670 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.922670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.922678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.924266 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.924670 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.924671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.924671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.924794 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.925013 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.925014 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.925014 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.925138 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.925217 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.925443 (+ 226us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.925521 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:17:07.926186 24090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: No bootstrap required, opened a new log
I20260809 02:17:07.928351 24090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.928512 24090 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:07.928555 24090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6619c5e9977f4d7d822775f20bd05475, State: Initialized, Role: FOLLOWER
I20260809 02:17:07.928686 24090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.929101 24090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:07.930220 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.917265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54752 (local address 127.9.40.125:44647)
0809 02:17:07.917307 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.917309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.917846 (+ 537us) server_negotiation.cc:407] Connection header received
0809 02:17:07.921381 (+ 3535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.921382 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.921397 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.921431 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.922875 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.923550 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.924576 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.924902 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.924944 (+ 42us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.926552 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.926553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.926553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.926561 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.928222 (+ 1661us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.928649 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.928650 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.928651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.928713 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.929003 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.929004 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.929004 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.929134 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.929704 (+ 570us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.929982 (+ 278us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.930126 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:17:07.930476 24093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.917266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53816 (local address 127.9.40.126:46209)
0809 02:17:07.917307 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.917309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.918169 (+ 860us) server_negotiation.cc:407] Connection header received
0809 02:17:07.918252 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.918253 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.918262 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.918289 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.918993 (+ 704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.919662 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.920813 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.921103 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.921152 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.927400 (+ 6248us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.927402 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.927402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.927411 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.929143 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.929548 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.929549 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.929549 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.929610 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.929843 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.929844 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.929844 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.929989 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.930055 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.930291 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.930384 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:17:07.931995 24293 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:17:07.932099 24293 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:17:07.932224 24282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:07.932287 24282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:07.932749 24018 master_runner.cc:186] Error getting master registration for 127.9.40.125:44647: OK, instance_id { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" instance_seqno: 1786241827722397 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:17:07.935809 24018 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:17:07.960660 24171 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } attempt: 3
Aug 09 02:17:07 dist-test-slave-qg60 krb5kdc[24003](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241827, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:17:07.969712 24175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.961035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46115 (local address 127.0.0.1:41924)
0809 02:17:07.961134 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:07.961146 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:07.961188 (+ 42us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:07.961504 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:07.961505 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:07.961711 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:07.961763 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.961765 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.962719 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.962721 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.963539 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:07.963542 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.964046 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.964047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:07.964110 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.964657 (+ 547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:07.964663 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:07.966443 (+ 1780us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:07.968468 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.968469 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.968469 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.968708 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.968985 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:07.968986 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:07.968987 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:07.969057 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:07.969346 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:07.969347 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:07.969399 (+ 52us) client_negotiation.cc:769] Sending connection context
0809 02:17:07.969472 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:17:07.969576 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260809 02:17:07.969693 24254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:07.961288 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41924 (local address 127.9.40.124:46115)
0809 02:17:07.961344 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:07.961347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:07.961356 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:07.961380 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:07.961380 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:07.961391 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:07.961446 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:07.961858 (+ 412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.962614 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.963649 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:07.963956 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:07.964033 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:07.966537 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:07.966539 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:07.966539 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:07.966547 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:07.968371 (+ 1824us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.968819 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.968820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.968820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.968884 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:07.969125 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:07.969126 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:07.969126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:07.969271 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:07.969343 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:07.969512 (+ 169us) server_negotiation.cc:299] Negotiation successful
0809 02:17:07.969593 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:17:07.971884 24171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: Bootstrap starting.
I20260809 02:17:07.972716 24171 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:07.973573 24171 log.cc:824] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:07.974545 24171 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: No bootstrap required, opened a new log
I20260809 02:17:07.976753 24171 raft_consensus.cc:348] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.976907 24171 raft_consensus.cc:374] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:07.976943 24171 raft_consensus.cc:729] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d5f6f91fe82a470c947429ffd53fc649, State: Initialized, Role: FOLLOWER
I20260809 02:17:07.977137 24171 consensus_queue.cc:260] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:07.977524 24171 sys_catalog.cc:564] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:07.977491 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:07.977701 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:17:07.979601 24307 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:17:07.979727 24307 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:17:07.982072 24098 master_runner.cc:263] No leader master found from master d5f6f91fe82a470c947429ffd53fc649
I20260809 02:17:07.982125 24098 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 02:17:08.060037 24041 tablet.cc:2369] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:08.127437 24202 tablet.cc:2369] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:08.220045 24121 tablet.cc:2369] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:08.378585 24296 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:08.378784 24296 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:08.379184 24296 leader_election.cc:288] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6619c5e9977f4d7d822775f20bd05475 (127.9.40.126:46209), a48b9366e5844839bcdc63dccb5da611 (127.9.40.124:46115)
I20260809 02:17:08.379477 24223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f6f91fe82a470c947429ffd53fc649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48b9366e5844839bcdc63dccb5da611" is_pre_election: true
I20260809 02:17:08.379491 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f6f91fe82a470c947429ffd53fc649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619c5e9977f4d7d822775f20bd05475" is_pre_election: true
I20260809 02:17:08.379622 24223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f6f91fe82a470c947429ffd53fc649 in term 0.
I20260809 02:17:08.379696 24064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d5f6f91fe82a470c947429ffd53fc649 in term 0.
I20260809 02:17:08.379868 24117 leader_election.cc:302] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a48b9366e5844839bcdc63dccb5da611, d5f6f91fe82a470c947429ffd53fc649; no voters:
I20260809 02:17:08.380049 24296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:08.380089 24296 raft_consensus.cc:482] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:08.380105 24296 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:08.380908 24296 raft_consensus.cc:504] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:08.381068 24296 leader_election.cc:288] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [CANDIDATE]: Term 1 election: Requested vote from peers 6619c5e9977f4d7d822775f20bd05475 (127.9.40.126:46209), a48b9366e5844839bcdc63dccb5da611 (127.9.40.124:46115)
I20260809 02:17:08.381232 24223 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f6f91fe82a470c947429ffd53fc649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a48b9366e5844839bcdc63dccb5da611"
I20260809 02:17:08.381227 24064 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "d5f6f91fe82a470c947429ffd53fc649" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6619c5e9977f4d7d822775f20bd05475"
I20260809 02:17:08.381314 24223 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:08.381319 24064 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:08.382030 24223 raft_consensus.cc:2454] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f6f91fe82a470c947429ffd53fc649 in term 1.
I20260809 02:17:08.382220 24064 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d5f6f91fe82a470c947429ffd53fc649 in term 1.
I20260809 02:17:08.382267 24117 leader_election.cc:302] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a48b9366e5844839bcdc63dccb5da611, d5f6f91fe82a470c947429ffd53fc649; no voters:
I20260809 02:17:08.382361 24296 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:08.382486 24296 raft_consensus.cc:686] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [term 1 LEADER]: Becoming Leader. State: Replica: d5f6f91fe82a470c947429ffd53fc649, State: Running, Role: LEADER
I20260809 02:17:08.382624 24296 consensus_queue.cc:237] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } }
I20260809 02:17:08.383054 24309 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5f6f91fe82a470c947429ffd53fc649. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.383154 24309 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:08.383338 24311 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:08.383543 24311 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:08.385082 24223 raft_consensus.cc:1261] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [term 1 FOLLOWER]: Refusing update from remote peer d5f6f91fe82a470c947429ffd53fc649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:08.385171 24064 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [term 1 FOLLOWER]: Refusing update from remote peer d5f6f91fe82a470c947429ffd53fc649: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:08.385460 24309 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:08.385564 24296 consensus_queue.cc:1035] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:08.386776 24282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5f6f91fe82a470c947429ffd53fc649. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.386875 24282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:08.386845 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: SysCatalogTable state changed. Reason: New leader d5f6f91fe82a470c947429ffd53fc649. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.386940 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:08.387714 24296 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.387737 24317 sys_catalog.cc:455] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.387816 24296 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:08.387835 24317 sys_catalog.cc:458] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:08.387867 24312 mvcc.cc:205] Tried to move back new op lower bound from 7316446529062719488 to 7316446529056096256. Current Snapshot: MvccSnapshot[applied={T|T < 7316446529062719488}]
I20260809 02:17:08.388149 24311 catalog_manager.cc:1347] Generated new cluster ID: 900818af999140178dd471daa381304f
I20260809 02:17:08.388200 24311 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:08.388190 24282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.388278 24282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:08.388368 24313 mvcc.cc:205] Tried to move back new op lower bound from 7316446529062719488 to 7316446529056096256. Current Snapshot: MvccSnapshot[applied={T|T < 7316446529062719488 or (T in {7316446529062719488})}]
I20260809 02:17:08.388715 24268 sys_catalog.cc:455] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d5f6f91fe82a470c947429ffd53fc649" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6619c5e9977f4d7d822775f20bd05475" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46209 } } peers { permanent_uuid: "d5f6f91fe82a470c947429ffd53fc649" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44647 } } peers { permanent_uuid: "a48b9366e5844839bcdc63dccb5da611" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46115 } } }
I20260809 02:17:08.388808 24268 sys_catalog.cc:458] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:17:08.396047 24311 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:08.396651 24311 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:08.404184 24311 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d5f6f91fe82a470c947429ffd53fc649: Generated new TSK 0
I20260809 02:17:08.404449 24311 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:17:08.921984 24279 catalog_manager.cc:1259] Loaded cluster ID: 900818af999140178dd471daa381304f
I20260809 02:17:08.922065 24279 catalog_manager.cc:1552] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: loading cluster ID for follower catalog manager: success
I20260809 02:17:08.922832 24279 catalog_manager.cc:1574] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: acquiring CA information for follower catalog manager: success
I20260809 02:17:08.923076 24279 catalog_manager.cc:1602] T 00000000000000000000000000000000 P a48b9366e5844839bcdc63dccb5da611: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:17:08.932513 24293 catalog_manager.cc:1259] Loaded cluster ID: 900818af999140178dd471daa381304f
I20260809 02:17:08.932592 24293 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: loading cluster ID for follower catalog manager: success
I20260809 02:17:08.933686 24293 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: acquiring CA information for follower catalog manager: success
I20260809 02:17:08.933966 24293 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6619c5e9977f4d7d822775f20bd05475: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:17:08.968554 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24018
I20260809 02:17:08.977622 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24098
I20260809 02:17:08.983757 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24179
2026-08-09T02:17:08Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (2812 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:17:09 dist-test-slave-qg60 krb5kdc[24329](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:17:09 dist-test-slave-qg60 krb5kdc[24329](info): set up 2 sockets
Aug 09 02:17:09 dist-test-slave-qg60 krb5kdc[24329](info): commencing operation
krb5kdc: starting...
W20260809 02:17:10.106451 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.098s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:17:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:17:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:17:10.240393 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45995
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43669
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:45995
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:10.323818 24343 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:10.324086 24343 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:10.324128 24343 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:10.325320 24343 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:10.325407 24343 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:10.325434 24343 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:10.325461 24343 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:10.325488 24343 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:10.327054 24343 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43669
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:45995
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:45995
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24343
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:10.327368 24343 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:10.327620 24343 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:10.332731 24351 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:10.332844 24349 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:10.332760 24348 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:10.332809 24343 server_base.cc:1034] running on GCE node
I20260809 02:17:10.333185 24343 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:10.333492 24343 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:10.334668 24343 hybrid_clock.cc:648] HybridClock initialized: now 1786241830334656 us; error 47 us; skew 500 ppm
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24329](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241830, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:10.337412 24343 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:10.337878 24343 webserver.cc:460] Webserver started at http://127.9.40.126:44967/ using document root <none> and password file <none>
I20260809 02:17:10.338125 24343 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:10.338195 24343 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:10.338340 24343 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:10.339382 24343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "056a463a538a4b2f9e9b5e961d83b340"
format_stamp: "Formatted at 2026-08-09 02:17:10 on dist-test-slave-qg60"
server_key: "dab97efab31071b6181e48476f1efb81"
server_key_iv: "580eb6e330f3a54eac46b91335813b7c"
server_key_version: "encryptionkey@0"
I20260809 02:17:10.339747 24343 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "056a463a538a4b2f9e9b5e961d83b340"
format_stamp: "Formatted at 2026-08-09 02:17:10 on dist-test-slave-qg60"
server_key: "dab97efab31071b6181e48476f1efb81"
server_key_iv: "580eb6e330f3a54eac46b91335813b7c"
server_key_version: "encryptionkey@0"
I20260809 02:17:10.341151 24343 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:17:10.341926 24358 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:10.342108 24343 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:10.342216 24343 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "056a463a538a4b2f9e9b5e961d83b340"
format_stamp: "Formatted at 2026-08-09 02:17:10 on dist-test-slave-qg60"
server_key: "dab97efab31071b6181e48476f1efb81"
server_key_iv: "580eb6e330f3a54eac46b91335813b7c"
server_key_version: "encryptionkey@0"
I20260809 02:17:10.342289 24343 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:10.373517 24343 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:10.375771 24343 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:10.375945 24343 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:10.380218 24343 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:45995
I20260809 02:17:10.380275 24413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:45995 every 8 connection(s)
I20260809 02:17:10.380648 24343 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:10.381618 24415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:10.383963 24415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340: Bootstrap starting.
I20260809 02:17:10.384569 24415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:10.384917 24415 log.cc:824] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:10.385703 24415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340: No bootstrap required, opened a new log
I20260809 02:17:10.386821 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24343
I20260809 02:17:10.386936 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:17:10.387132 9377 external_mini_cluster.cc:1428] Setting key f09354d0993a5b9c3234626d4534d1ab
I20260809 02:17:10.387928 24415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } }
I20260809 02:17:10.388111 24415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:10.388164 24415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 056a463a538a4b2f9e9b5e961d83b340, State: Initialized, Role: FOLLOWER
I20260809 02:17:10.388334 24415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } }
I20260809 02:17:10.388425 24415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:10.388459 24415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:10.388525 24415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:10.389631 24415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } }
I20260809 02:17:10.389721 24415 leader_election.cc:302] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 056a463a538a4b2f9e9b5e961d83b340; no voters:
I20260809 02:17:10.389891 24415 leader_election.cc:288] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:17:10.389940 24421 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:10.390115 24421 raft_consensus.cc:686] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [term 1 LEADER]: Becoming Leader. State: Replica: 056a463a538a4b2f9e9b5e961d83b340, State: Running, Role: LEADER
I20260809 02:17:10.390172 24415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:10.390255 24421 consensus_queue.cc:237] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } }
I20260809 02:17:10.390657 24421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 056a463a538a4b2f9e9b5e961d83b340. Latest consensus state: current_term: 1 leader_uuid: "056a463a538a4b2f9e9b5e961d83b340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } } }
I20260809 02:17:10.390797 24421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:10.391361 24430 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:10.391782 24423 sys_catalog.cc:455] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "056a463a538a4b2f9e9b5e961d83b340" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "056a463a538a4b2f9e9b5e961d83b340" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 45995 } } }
I20260809 02:17:10.391863 24430 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:10.391884 24423 sys_catalog.cc:458] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:10.393501 24430 catalog_manager.cc:1347] Generated new cluster ID: f2346184fe404f07afb66b093f9279fe
I20260809 02:17:10.393563 24430 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24329](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241830, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:10.400002 24420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:10.387677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49266 (local address 127.9.40.126:45995)
0809 02:17:10.387858 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:10.387865 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:10.387891 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:17:10.387982 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:10.387988 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:10.388029 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:10.388209 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:10.389075 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:10.390222 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:10.391550 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:10.392153 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:10.392276 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:10.395130 (+ 2854us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:10.395134 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:10.395134 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:10.395146 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:10.398083 (+ 2937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:10.398477 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:10.398480 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:10.398481 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:10.398604 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:10.398796 (+ 192us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:10.398797 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:10.398797 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:10.399083 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:10.399219 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:10.399650 (+ 431us) server_negotiation.cc:299] Negotiation successful
0809 02:17:10.399786 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":96,"threads_started":1}
I20260809 02:17:10.401510 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:17:10.407086 24430 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:10.407850 24430 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:10.423808 24430 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 056a463a538a4b2f9e9b5e961d83b340: Generated new TSK 0
I20260809 02:17:10.424045 24430 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:17:10.455546 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24343
2026-08-09T02:17:10Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1467 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24443](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24443](info): set up 2 sockets
Aug 09 02:17:10 dist-test-slave-qg60 krb5kdc[24443](info): commencing operation
krb5kdc: starting...
W20260809 02:17:11.560822 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.085s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:17:11 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:17:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:17:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:17:11.690145 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:11.782645 24456 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:11.782917 24456 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:11.782968 24456 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:11.784204 24456 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:11.784276 24456 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:11.784299 24456 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:11.784319 24456 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:11.784349 24456 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:11.785897 24456 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24456
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:11.786273 24456 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:11.786501 24456 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:11.791517 24463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:11.791510 24461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:11.791510 24465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:11.791875 24456 server_base.cc:1034] running on GCE node
I20260809 02:17:11.792142 24456 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:11.792397 24456 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:11.793576 24456 hybrid_clock.cc:648] HybridClock initialized: now 1786241831793557 us; error 45 us; skew 500 ppm
Aug 09 02:17:11 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241831, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:11.796103 24456 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:11.796568 24456 webserver.cc:460] Webserver started at http://127.9.40.126:34853/ using document root <none> and password file <none>
I20260809 02:17:11.796825 24456 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:11.796881 24456 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:11.796975 24456 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:11.797999 24456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "5972ac8f885d44e391823127102fc8fd"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "9d9256e664e78fb39f49f4c44c662079"
server_key_iv: "5ad6fedeb5e05454e5549d0b831dbef4"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.798311 24456 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "5972ac8f885d44e391823127102fc8fd"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "9d9256e664e78fb39f49f4c44c662079"
server_key_iv: "5ad6fedeb5e05454e5549d0b831dbef4"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.799620 24456 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:17:11.800455 24472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:11.800700 24456 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:11.800848 24456 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "5972ac8f885d44e391823127102fc8fd"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "9d9256e664e78fb39f49f4c44c662079"
server_key_iv: "5ad6fedeb5e05454e5549d0b831dbef4"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.800930 24456 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:11.815056 24456 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:11.817324 24456 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:11.817494 24456 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:11.822156 24456 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43661
I20260809 02:17:11.822234 24531 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43661 every 8 connection(s)
I20260809 02:17:11.822561 24456 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:11.823617 24532 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:11.825992 24532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap starting.
I20260809 02:17:11.826407 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24456
I20260809 02:17:11.826498 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:17:11.826699 24532 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:11.826710 9377 external_mini_cluster.cc:1428] Setting key b7b87ccc4ecda599b563deee664c0a53
I20260809 02:17:11.827018 24532 log.cc:824] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:11.827922 24532 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: No bootstrap required, opened a new log
I20260809 02:17:11.829849 24532 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:11.830016 24532 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:11.830060 24532 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Initialized, Role: FOLLOWER
I20260809 02:17:11.830211 24532 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:11.830307 24532 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:11.830353 24532 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:11.830403 24532 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:11.830981 24532 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:11.831106 24532 leader_election.cc:302] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5972ac8f885d44e391823127102fc8fd; no voters:
I20260809 02:17:11.831323 24532 leader_election.cc:288] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:17:11.831348 24538 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:11.831538 24538 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 LEADER]: Becoming Leader. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Running, Role: LEADER
I20260809 02:17:11.831655 24532 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:11.831661 24538 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:11.833104 24539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5972ac8f885d44e391823127102fc8fd. Latest consensus state: current_term: 1 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:11.833197 24539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:11.833263 24540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:11.833323 24540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:11.833509 24550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Aug 09 02:17:11 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241831, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:11.834108 24550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:11.835844 24550 catalog_manager.cc:1347] Generated new cluster ID: 00bafdd6337e4ef0b7662ff57e004887
I20260809 02:17:11.835904 24550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:11.838172 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:11.827253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.9.40.126:43661)
0809 02:17:11.827444 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:11.827454 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:11.827488 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:17:11.827979 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:11.827988 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:11.828061 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:11.828212 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:11.828930 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:11.829821 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:11.830809 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:11.831316 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:11.831434 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:11.834540 (+ 3106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:11.834545 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:11.834545 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:11.834559 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:11.836517 (+ 1958us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:11.836955 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:11.836958 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:11.836958 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:11.837052 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:11.837280 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:11.837281 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:11.837281 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:11.837519 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:11.837700 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:11.837790 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:17:11.837962 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":82,"threads_started":1}
I20260809 02:17:11.854745 24550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:11.855504 24550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:11.865123 24550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Generated new TSK 0
I20260809 02:17:11.865350 24550 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:17:11.895691 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:11.980410 24561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:11.980661 24561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:11.980687 24561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:11.981892 24561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:11.981941 24561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:11.981969 24561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:11.983422 24561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:11.983624 24561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:11.983816 24561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:11.989053 24569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:11.989115 24561 server_base.cc:1034] running on GCE node
W20260809 02:17:11.989059 24567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:11.989069 24566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:11.989498 24561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:11.989766 24561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:11.990916 24561 hybrid_clock.cc:648] HybridClock initialized: now 1786241831990897 us; error 43 us; skew 500 ppm
Aug 09 02:17:11 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241831, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:11.993590 24561 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:11.994102 24561 webserver.cc:460] Webserver started at http://127.9.40.65:36937/ using document root <none> and password file <none>
I20260809 02:17:11.994360 24561 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:11.994431 24561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:11.994581 24561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:11.995667 24561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "ea164fa018aeddd570668d52a390c852"
server_key_iv: "4347d5404545297152878438c444fae0"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.996037 24561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "ea164fa018aeddd570668d52a390c852"
server_key_iv: "4347d5404545297152878438c444fae0"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.997512 24561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260809 02:17:11.998297 24576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:11.998606 24561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:11.998719 24561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "ea164fa018aeddd570668d52a390c852"
server_key_iv: "4347d5404545297152878438c444fae0"
server_key_version: "encryptionkey@0"
I20260809 02:17:11.998798 24561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.018114 24561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.020146 24561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.020323 24561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.020593 24561 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:12.021013 24561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:12.021070 24561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.021108 24561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:12.021152 24561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.027508 24561 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33385
I20260809 02:17:12.027616 24700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33385 every 8 connection(s)
I20260809 02:17:12.027964 24561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:12.031303 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24561
I20260809 02:17:12.031448 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:17:12.031677 9377 external_mini_cluster.cc:1428] Setting key c03c658a3284f7ff5a4ca77889bae278
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241831, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:12.040120 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.028855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60229 (local address 127.9.40.126:43661)
0809 02:17:12.028932 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.028934 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.029211 (+ 277us) server_negotiation.cc:407] Connection header received
0809 02:17:12.029785 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.029786 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.029800 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.029866 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:12.030887 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.031566 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.033336 (+ 1770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.033793 (+ 457us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.033888 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.036660 (+ 2772us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:12.036662 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:12.036663 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:12.036673 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:12.038451 (+ 1778us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.038936 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.038938 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.038938 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.039013 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.039314 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.039315 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.039316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.039464 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:12.039552 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.039870 (+ 318us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.040005 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:17:12.040365 24703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.028883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:60229)
0809 02:17:12.029074 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.029114 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.029666 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.029943 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.029947 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.030568 (+ 621us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:12.030777 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.030782 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.031701 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.031702 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.033218 (+ 1516us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.033221 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.033975 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.033976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.034148 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.034879 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:12.034891 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:12.036537 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:12.038562 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.038572 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.038575 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.038844 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.039092 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.039093 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.039093 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.039225 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.039542 (+ 317us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:12.039551 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:12.039720 (+ 169us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.039848 (+ 128us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.040070 (+ 222us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260809 02:17:12.041010 24701 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:12.041158 24701 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:17:12.041386 24701 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:12.042047 24493 ts_manager.cc:194] Registered new tserver with Master: eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385)
I20260809 02:17:12.042801 24493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:60229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:17:12.060233 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:12.144565 24706 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:12.144817 24706 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:12.144842 24706 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:12.145959 24706 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:12.145994 24706 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:12.146021 24706 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:12.147432 24706 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24706
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:12.147630 24706 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:12.147820 24706 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:12.152992 24711 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.153123 24712 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.153010 24715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.153276 24706 server_base.cc:1034] running on GCE node
I20260809 02:17:12.153568 24706 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:12.153832 24706 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:12.155028 24706 hybrid_clock.cc:648] HybridClock initialized: now 1786241832155007 us; error 45 us; skew 500 ppm
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:12.157792 24706 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:12.158264 24706 webserver.cc:460] Webserver started at http://127.9.40.66:41463/ using document root <none> and password file <none>
I20260809 02:17:12.158535 24706 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:12.158607 24706 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:12.158756 24706 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:12.159811 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "2e343b58bf74430fa8ec205171ff9d5d"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "99d1ebcd25a46906e4631cbc4a074349"
server_key_iv: "b38f7cf7f4ed7fb987c717670ddf678d"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.160173 24706 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "2e343b58bf74430fa8ec205171ff9d5d"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "99d1ebcd25a46906e4631cbc4a074349"
server_key_iv: "b38f7cf7f4ed7fb987c717670ddf678d"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.161651 24706 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.002s
I20260809 02:17:12.162623 24722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.162837 24706 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:12.162952 24706 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "2e343b58bf74430fa8ec205171ff9d5d"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "99d1ebcd25a46906e4631cbc4a074349"
server_key_iv: "b38f7cf7f4ed7fb987c717670ddf678d"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.163048 24706 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.189096 24706 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.191187 24706 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.191354 24706 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.191608 24706 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:12.191919 24706 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:12.191973 24706 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.192019 24706 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:12.192044 24706 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.198143 24706 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39129
I20260809 02:17:12.198256 24844 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39129 every 8 connection(s)
I20260809 02:17:12.198545 24706 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:12.206425 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24706
I20260809 02:17:12.206511 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:17:12.206717 9377 external_mini_cluster.cc:1428] Setting key b3fbc1e70f8e432cce493696602d6963
I20260809 02:17:12.208899 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.199362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42011 (local address 127.9.40.126:43661)
0809 02:17:12.199413 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.199416 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.199603 (+ 187us) server_negotiation.cc:407] Connection header received
0809 02:17:12.200117 (+ 514us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.200118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.200131 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.200209 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:12.201239 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.201855 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.202940 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.203154 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.203216 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.205504 (+ 2288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:12.205505 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:12.205505 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:12.205513 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:12.207281 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.207799 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.207799 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.207800 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.207856 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.208174 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.208174 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.208175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.208285 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:12.208340 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.208695 (+ 355us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.208828 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:17:12.209239 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.199320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:42011)
0809 02:17:12.199473 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.199515 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.199993 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.200262 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.200265 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.200925 (+ 660us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:12.201130 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.201134 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.201954 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.201954 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.202850 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.202853 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.203233 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.203234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.203384 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.203894 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:12.203902 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:12.205412 (+ 1510us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:12.207384 (+ 1972us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.207390 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.207391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.207722 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.207925 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.207926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.207926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.208097 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.208352 (+ 255us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:12.208357 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:12.208531 (+ 174us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.208657 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.208867 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":83,"threads_started":1}
I20260809 02:17:12.209806 24845 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:12.209895 24845 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:12.210104 24845 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:12.210510 24493 ts_manager.cc:194] Registered new tserver with Master: 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129)
I20260809 02:17:12.210852 24493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:42011
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:17:12.235934 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:12.323242 24850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:12.323530 24850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:12.323585 24850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:12.325079 24850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:12.325156 24850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:12.325206 24850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:12.326902 24850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:12.327236 24850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:12.327524 24850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:12.333264 24855 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.333269 24856 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.333304 24858 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.333719 24850 server_base.cc:1034] running on GCE node
I20260809 02:17:12.333953 24850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:12.334241 24850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:12.335397 24850 hybrid_clock.cc:648] HybridClock initialized: now 1786241832335373 us; error 37 us; skew 500 ppm
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:12.338107 24850 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:12.338619 24850 webserver.cc:460] Webserver started at http://127.9.40.67:35027/ using document root <none> and password file <none>
I20260809 02:17:12.338868 24850 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:12.338913 24850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:12.339066 24850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:12.340150 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "6ad2724437bb43c0a3f673d0182202c7"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "3b3c81ad684fd88682d279819930de6d"
server_key_iv: "54bff072fd0eb63c56b4b19b3219e251"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.340517 24850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "6ad2724437bb43c0a3f673d0182202c7"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "3b3c81ad684fd88682d279819930de6d"
server_key_iv: "54bff072fd0eb63c56b4b19b3219e251"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.342053 24850 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:12.342810 24865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.342981 24850 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:17:12.343084 24850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "6ad2724437bb43c0a3f673d0182202c7"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "3b3c81ad684fd88682d279819930de6d"
server_key_iv: "54bff072fd0eb63c56b4b19b3219e251"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.343168 24850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.358891 24850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.360922 24850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.361083 24850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.361323 24850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:12.361654 24850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:12.361711 24850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.361763 24850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:12.361785 24850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.368608 24850 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35139
I20260809 02:17:12.368690 24989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35139 every 8 connection(s)
I20260809 02:17:12.369038 24850 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:12.372043 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 24850
I20260809 02:17:12.372164 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:17:12.372324 9377 external_mini_cluster.cc:1428] Setting key 1116ab874265f2aca8f853abb31af447
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:12.379189 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.369920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58693 (local address 127.9.40.126:43661)
0809 02:17:12.369971 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.369974 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.370238 (+ 264us) server_negotiation.cc:407] Connection header received
0809 02:17:12.370714 (+ 476us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.370715 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.370727 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.370800 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:12.371790 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.372621 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.373815 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.374077 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.374177 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.376438 (+ 2261us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:12.376439 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:12.376439 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:12.376446 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:12.377935 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.378348 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.378349 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.378350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.378408 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.378634 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.378634 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.378635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.378746 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:12.378799 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.379039 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.379118 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:17:12.379386 24992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.369905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:58693)
0809 02:17:12.370062 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.370105 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.370600 (+ 495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.370856 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.370860 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.371465 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:12.371669 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.371673 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.372770 (+ 1097us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.372771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.373688 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.373691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.374190 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.374191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.374337 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.374853 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:12.374862 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:12.376339 (+ 1477us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:12.378019 (+ 1680us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.378025 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.378027 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.378271 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.378474 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.378475 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.378475 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.378575 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.378804 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:12.378808 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:12.378927 (+ 119us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.379010 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.379157 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260809 02:17:12.379880 24990 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:12.379977 24990 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:12.380122 24990 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:12.380475 24493 ts_manager.cc:194] Registered new tserver with Master: 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
I20260809 02:17:12.380883 24493 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:58693
I20260809 02:17:12.383940 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:12.391343 24536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.385337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35132 (local address 127.9.40.126:43661)
0809 02:17:12.385402 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.385403 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.385549 (+ 146us) server_negotiation.cc:407] Connection header received
0809 02:17:12.385622 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.385623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.385643 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.385688 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:12.386237 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.386708 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.387422 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.387596 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.387660 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.388852 (+ 1192us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:12.388853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:12.388854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:12.388860 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:12.390148 (+ 1288us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.390463 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.390464 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.390464 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.390514 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.390686 (+ 172us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.390686 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.390686 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.390821 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:12.390901 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.391210 (+ 309us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.391279 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:17:12.392995 24493 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35132:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:12.394598 24493 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:12.401695 25001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397094 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39129 (local address 127.0.0.1:33768)
0809 02:17:12.397254 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.397297 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.397363 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.397764 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.397767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.397773 (+ 6us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.397860 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.397861 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.399000 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.399001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.400186 (+ 1185us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.400189 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.401325 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.401326 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.401409 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.401432 (+ 23us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.401531 (+ 99us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.401598 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":113,"threads_started":1}
I20260809 02:17:12.401777 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397706 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34414 (local address 127.9.40.67:35139)
0809 02:17:12.398212 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.398217 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.398227 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:17:12.398261 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.398263 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.398277 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.398347 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.398559 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.399627 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.400798 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.401410 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.401492 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.401546 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.401648 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.401689 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":96,"threads_started":1}
I20260809 02:17:12.401795 25004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35139 (local address 127.0.0.1:34414)
0809 02:17:12.397719 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.397729 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.397779 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.398424 (+ 645us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.398424 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.398427 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.398473 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.398475 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.399737 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.399738 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.400669 (+ 931us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.400672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.401502 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.401503 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.401561 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.401563 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.401616 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.401649 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":30,"thread_start_us":111,"threads_started":1}
I20260809 02:17:12.402190 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397223 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33768 (local address 127.9.40.66:39129)
0809 02:17:12.397425 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.397430 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.397442 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:17:12.397489 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.397491 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.397504 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.397649 (+ 145us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.397939 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.398880 (+ 941us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.400302 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.401248 (+ 946us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.401323 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.401390 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.402060 (+ 670us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.402106 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":130,"threads_started":1}
I20260809 02:17:12.402716 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.0.0.1:46760)
0809 02:17:12.397495 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.397504 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.397543 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.398255 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.398256 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.398258 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.398318 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.398320 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.399751 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.399752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.401139 (+ 1387us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.401142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.402497 (+ 1355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.402498 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.402565 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.402569 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.402602 (+ 33us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.402647 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":90,"threads_started":1}
I20260809 02:17:12.402609 24919 tablet_service.cc:1467] Processing CreateTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 (DEFAULT_TABLE table=test-table [id=e46d807d353b4f91b025e96598774f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:12.402858 24774 tablet_service.cc:1467] Processing CreateTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 (DEFAULT_TABLE table=test-table [id=e46d807d353b4f91b025e96598774f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:12.403026 24919 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dcc26b96af9423fabaf7113f33a84e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:12.403119 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.397879 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46760 (local address 127.9.40.65:33385)
0809 02:17:12.398051 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.398055 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.398064 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:12.398108 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.398110 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.398121 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.398398 (+ 277us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.398469 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.399595 (+ 1126us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.401222 (+ 1627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.402396 (+ 1174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.402903 (+ 507us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.402970 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.402997 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.403041 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":97,"threads_started":1}
I20260809 02:17:12.403263 24774 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dcc26b96af9423fabaf7113f33a84e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:12.405989 25007 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Bootstrap starting.
I20260809 02:17:12.406517 24628 tablet_service.cc:1467] Processing CreateTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 (DEFAULT_TABLE table=test-table [id=e46d807d353b4f91b025e96598774f6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:12.406769 25007 tablet_bootstrap.cc:654] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:12.406860 24628 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4dcc26b96af9423fabaf7113f33a84e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:12.407382 25007 log.cc:824] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:12.408278 25007 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: No bootstrap required, opened a new log
I20260809 02:17:12.408360 25007 ts_tablet_manager.cc:1397] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:12.409787 25010 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap starting.
I20260809 02:17:12.409916 25008 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Bootstrap starting.
I20260809 02:17:12.409827 25007 raft_consensus.cc:348] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.410013 25007 raft_consensus.cc:374] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:12.410060 25007 raft_consensus.cc:729] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad2724437bb43c0a3f673d0182202c7, State: Initialized, Role: FOLLOWER
I20260809 02:17:12.410207 25007 consensus_queue.cc:260] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.410490 25010 tablet_bootstrap.cc:654] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:12.410524 24990 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
I20260809 02:17:12.410534 25008 tablet_bootstrap.cc:654] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:12.410806 25007 ts_tablet_manager.cc:1428] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:17:12.411118 25008 log.cc:824] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:12.411118 25010 log.cc:824] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:12.412114 25008 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: No bootstrap required, opened a new log
I20260809 02:17:12.412170 25008 ts_tablet_manager.cc:1397] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:12.412223 25010 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: No bootstrap required, opened a new log
I20260809 02:17:12.412294 25010 ts_tablet_manager.cc:1397] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:12.413591 25008 raft_consensus.cc:348] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.413726 25008 raft_consensus.cc:374] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:12.413748 25008 raft_consensus.cc:729] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e343b58bf74430fa8ec205171ff9d5d, State: Initialized, Role: FOLLOWER
I20260809 02:17:12.413697 25010 raft_consensus.cc:348] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.413902 25010 raft_consensus.cc:374] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:12.413951 25010 raft_consensus.cc:729] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb1bdadd0fb447edb5ab4013d86ceaad, State: Initialized, Role: FOLLOWER
I20260809 02:17:12.413892 25008 consensus_queue.cc:260] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.414063 25010 consensus_queue.cc:260] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.414207 24845 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
I20260809 02:17:12.414224 25008 ts_tablet_manager.cc:1428] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:17:12.414363 25010 ts_tablet_manager.cc:1428] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:12.414444 24701 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
W20260809 02:17:12.449141 24846 tablet.cc:2369] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:12.472709 25014 raft_consensus.cc:482] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:12.473052 25014 raft_consensus.cc:504] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.473521 25014 leader_election.cc:288] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385), 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
I20260809 02:17:12.477492 25005 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.473947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50421 (local address 127.9.40.67:35139)
0809 02:17:12.474083 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.474085 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.474175 (+ 90us) server_negotiation.cc:407] Connection header received
0809 02:17:12.474239 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.474240 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.474253 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.474302 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.474404 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.475097 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.476483 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.477105 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.477180 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.477190 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.477382 (+ 192us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.477421 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260809 02:17:12.477527 25006 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.473804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:49201 (local address 127.9.40.65:33385)
0809 02:17:12.473881 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.473884 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.473915 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:17:12.473944 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.473945 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.473960 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.474007 (+ 47us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.474208 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.475007 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.476483 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.477098 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.477180 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.477191 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.477382 (+ 191us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.477421 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260809 02:17:12.477522 24847 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.473720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.9.40.66:49201)
0809 02:17:12.473791 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.473805 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.473873 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.474038 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.474039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.474042 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.474126 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.474128 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.475095 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.475096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.476363 (+ 1267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.476367 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.477207 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.477208 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.477295 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.477296 (+ 1us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.477358 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.477402 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":40}
I20260809 02:17:12.477530 25016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.473877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35139 (local address 127.9.40.66:50421)
0809 02:17:12.474083 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.474095 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.474172 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.474304 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.474305 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.474306 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.474353 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.474354 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.475181 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.475181 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.476364 (+ 1183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.476367 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.477208 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.477208 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.477291 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.477296 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.477358 (+ 62us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.477402 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":7,"thread_start_us":125,"threads_started":1}
I20260809 02:17:12.478050 24653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dcc26b96af9423fabaf7113f33a84e0" candidate_uuid: "2e343b58bf74430fa8ec205171ff9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" is_pre_election: true
I20260809 02:17:12.478153 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dcc26b96af9423fabaf7113f33a84e0" candidate_uuid: "2e343b58bf74430fa8ec205171ff9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad2724437bb43c0a3f673d0182202c7" is_pre_election: true
I20260809 02:17:12.478242 24653 raft_consensus.cc:2454] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e343b58bf74430fa8ec205171ff9d5d in term 0.
I20260809 02:17:12.478319 24941 raft_consensus.cc:2454] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e343b58bf74430fa8ec205171ff9d5d in term 0.
I20260809 02:17:12.478484 24725 leader_election.cc:302] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e343b58bf74430fa8ec205171ff9d5d, eb1bdadd0fb447edb5ab4013d86ceaad; no voters:
I20260809 02:17:12.478658 25014 raft_consensus.cc:2781] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:12.478735 25014 raft_consensus.cc:482] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:12.478753 25014 raft_consensus.cc:3037] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:12.479563 25014 raft_consensus.cc:504] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.479676 25014 leader_election.cc:288] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [CANDIDATE]: Term 1 election: Requested vote from peers eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385), 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
I20260809 02:17:12.479848 24653 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dcc26b96af9423fabaf7113f33a84e0" candidate_uuid: "2e343b58bf74430fa8ec205171ff9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
I20260809 02:17:12.479908 24941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4dcc26b96af9423fabaf7113f33a84e0" candidate_uuid: "2e343b58bf74430fa8ec205171ff9d5d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6ad2724437bb43c0a3f673d0182202c7"
I20260809 02:17:12.479934 24653 raft_consensus.cc:3037] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:12.479986 24941 raft_consensus.cc:3037] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:12.480912 24653 raft_consensus.cc:2454] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e343b58bf74430fa8ec205171ff9d5d in term 1.
I20260809 02:17:12.480911 24941 raft_consensus.cc:2454] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e343b58bf74430fa8ec205171ff9d5d in term 1.
I20260809 02:17:12.481114 24725 leader_election.cc:302] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e343b58bf74430fa8ec205171ff9d5d, 6ad2724437bb43c0a3f673d0182202c7; no voters:
I20260809 02:17:12.481215 25014 raft_consensus.cc:2781] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:12.481381 25014 raft_consensus.cc:686] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 LEADER]: Becoming Leader. State: Replica: 2e343b58bf74430fa8ec205171ff9d5d, State: Running, Role: LEADER
I20260809 02:17:12.481485 25014 consensus_queue.cc:237] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.482358 24493 catalog_manager.cc:5495] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66). New cstate: current_term: 1 leader_uuid: "2e343b58bf74430fa8ec205171ff9d5d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } health_report { overall_health: UNKNOWN } } }
I20260809 02:17:12.496680 25002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.494102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33780 (local address 127.9.40.66:39129)
0809 02:17:12.494169 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.494171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.494182 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:12.494257 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.494258 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.494286 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.494365 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:12.494543 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.495017 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.495775 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.495970 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.496112 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.496287 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:12.496401 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:12.496488 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.496544 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.496565 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:17:12.498597 24941 raft_consensus.cc:1261] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Refusing update from remote peer 2e343b58bf74430fa8ec205171ff9d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:12.498641 24653 raft_consensus.cc:1261] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Refusing update from remote peer 2e343b58bf74430fa8ec205171ff9d5d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:12.498965 25014 consensus_queue.cc:1035] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:12.499131 25017 consensus_queue.cc:1035] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:12.502216 25022 mvcc.cc:205] Tried to move back new op lower bound from 7316446545911128064 to 7316446545844715520. Current Snapshot: MvccSnapshot[applied={T|T < 7316446545911128064}]
I20260809 02:17:12.502915 25021 mvcc.cc:205] Tried to move back new op lower bound from 7316446545911128064 to 7316446545844715520. Current Snapshot: MvccSnapshot[applied={T|T < 7316446545911128064}]
I20260809 02:17:12.504961 24493 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:12.505064 24493 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35132:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:12.505843 24493 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Sending DeleteTablet for 3 replicas of tablet 4dcc26b96af9423fabaf7113f33a84e0
I20260809 02:17:12.506209 24774 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:33768
I20260809 02:17:12.506225 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24561
I20260809 02:17:12.506307 24628 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:46760
I20260809 02:17:12.506925 24919 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34414
I20260809 02:17:12.507606 25034 tablet_replica.cc:330] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: stopping tablet replica
I20260809 02:17:12.507740 25034 raft_consensus.cc:2229] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:12.507885 25034 raft_consensus.cc:2258] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:12.508749 25034 ts_tablet_manager.cc:1905] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:12.510443 25033 tablet_replica.cc:330] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: stopping tablet replica
I20260809 02:17:12.510613 25033 raft_consensus.cc:2229] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:12.510776 25033 raft_consensus.cc:2258] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:12.510980 25034 ts_tablet_manager.cc:1918] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:12.511066 25034 log.cc:1196] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/4dcc26b96af9423fabaf7113f33a84e0
I20260809 02:17:12.511389 25033 ts_tablet_manager.cc:1905] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:12.511400 25034 ts_tablet_manager.cc:1939] T 4dcc26b96af9423fabaf7113f33a84e0 P 6ad2724437bb43c0a3f673d0182202c7: Deleting consensus metadata
I20260809 02:17:12.511898 24475 catalog_manager.cc:4841] TS 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139): tablet 4dcc26b96af9423fabaf7113f33a84e0 (table test-table [id=e46d807d353b4f91b025e96598774f6f]) successfully deleted
I20260809 02:17:12.513569 25033 ts_tablet_manager.cc:1918] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:12.513651 25033 log.cc:1196] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/4dcc26b96af9423fabaf7113f33a84e0
I20260809 02:17:12.513937 25033 ts_tablet_manager.cc:1939] T 4dcc26b96af9423fabaf7113f33a84e0 P 2e343b58bf74430fa8ec205171ff9d5d: Deleting consensus metadata
I20260809 02:17:12.514349 24475 catalog_manager.cc:4841] TS 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129): tablet 4dcc26b96af9423fabaf7113f33a84e0 (table test-table [id=e46d807d353b4f91b025e96598774f6f]) successfully deleted
W20260809 02:17:12.516486 24475 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:33385 (error 108)
I20260809 02:17:12.517127 25003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.516972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.0.0.1:46770)
0809 02:17:12.517036 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.517088 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260809 02:17:12.517120 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24706
W20260809 02:17:12.520016 24475 catalog_manager.cc:4584] TS eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4dcc26b96af9423fabaf7113f33a84e0: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33385: connect: Connection refused (error 111)
I20260809 02:17:12.524346 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24850
I20260809 02:17:12.530212 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 24456
I20260809 02:17:12.538102 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--webserver_interface=127.9.40.126
--webserver_port=34853
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:12.621025 25036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:12.621284 25036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:12.621321 25036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:12.622483 25036 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:12.622555 25036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:12.622578 25036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:12.622601 25036 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:12.622632 25036 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:12.624164 25036 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=34853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:12.624456 25036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:12.624702 25036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:12.636381 25042 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.636381 25045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.636441 25043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.636694 25036 server_base.cc:1034] running on GCE node
I20260809 02:17:12.636996 25036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:12.637341 25036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:12.638511 25036 hybrid_clock.cc:648] HybridClock initialized: now 1786241832638485 us; error 34 us; skew 500 ppm
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:12.645985 25036 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:12.646531 25036 webserver.cc:460] Webserver started at http://127.9.40.126:34853/ using document root <none> and password file <none>
I20260809 02:17:12.646790 25036 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:12.646867 25036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:12.648340 25036 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:17:12.649120 25052 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.649335 25036 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:12.649477 25036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "5972ac8f885d44e391823127102fc8fd"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "9d9256e664e78fb39f49f4c44c662079"
server_key_iv: "5ad6fedeb5e05454e5549d0b831dbef4"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.649955 25036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.657577 25036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.659437 25036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.659606 25036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.663810 25036 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43661
I20260809 02:17:12.663841 25109 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43661 every 8 connection(s)
I20260809 02:17:12.664242 25036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:12.666534 25110 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:17:12.667160 25110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap starting.
I20260809 02:17:12.673659 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25036
I20260809 02:17:12.673955 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:33385
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=36937
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:12.676095 25110 log.cc:824] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:12.678246 25110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:12.678599 25110 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap complete.
I20260809 02:17:12.681032 25110 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:12.681265 25110 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Initialized, Role: FOLLOWER
I20260809 02:17:12.681440 25110 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:12.681521 25110 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:12.681582 25110 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:12.681655 25110 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:17:12.682747 25110 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:12.682864 25110 leader_election.cc:302] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5972ac8f885d44e391823127102fc8fd; no voters:
I20260809 02:17:12.683085 25110 leader_election.cc:288] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:17:12.683122 25115 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:17:12.683235 25115 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 LEADER]: Becoming Leader. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Running, Role: LEADER
I20260809 02:17:12.683372 25110 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:12.683387 25115 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:12.683761 25116 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:12.683759 25117 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5972ac8f885d44e391823127102fc8fd. Latest consensus state: current_term: 2 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:12.683954 25116 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:12.684041 25117 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:12.684332 25125 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:12.685333 25125 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:12.685489 25125 catalog_manager.cc:1259] Loaded cluster ID: 00bafdd6337e4ef0b7662ff57e004887
I20260809 02:17:12.685750 25125 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:12.685993 25125 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260809 02:17:12.688771 25125 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260809 02:17:12.697866 25125 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:12.698117 25125 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Loaded TSK: 0
I20260809 02:17:12.698439 25125 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:17:12.792697 25112 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:12.793016 25112 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:12.793058 25112 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:12.794763 25112 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:12.794842 25112 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:12.794884 25112 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:12.797156 25112 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:33385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=36937
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25112
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:12.797420 25112 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:12.797705 25112 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:12.803375 25141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.803551 25112 server_base.cc:1034] running on GCE node
W20260809 02:17:12.803370 25142 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.803472 25144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.803918 25112 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:12.804185 25112 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:12.805344 25112 hybrid_clock.cc:648] HybridClock initialized: now 1786241832805331 us; error 34 us; skew 500 ppm
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:12.807884 25112 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:12.808353 25112 webserver.cc:460] Webserver started at http://127.9.40.65:36937/ using document root <none> and password file <none>
I20260809 02:17:12.808588 25112 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:12.808676 25112 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:12.810196 25112 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:12.810948 25152 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.811120 25112 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:12.811213 25112 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "ea164fa018aeddd570668d52a390c852"
server_key_iv: "4347d5404545297152878438c444fae0"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.811578 25112 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.823406 25112 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.825793 25112 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.825949 25112 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.826220 25112 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:12.826696 25161 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:17:12.828814 25112 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:17:12.828886 25112 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:17:12.828943 25112 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:17:12.829584 25112 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:17:12.829646 25112 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:12.829720 25161 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap starting.
I20260809 02:17:12.837648 25112 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33385
I20260809 02:17:12.837738 25273 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33385 every 8 connection(s)
I20260809 02:17:12.838304 25112 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:12.840363 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25112
I20260809 02:17:12.840679 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:39129
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=41463
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:12.841909 25161 log.cc:824] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:12.846016 25161 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:12.846877 25161 tablet_bootstrap.cc:492] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap complete.
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:12.848428 25161 ts_tablet_manager.cc:1397] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent bootstrapping tablet: real 0.019s user 0.006s sys 0.010s
I20260809 02:17:12.850652 25161 raft_consensus.cc:348] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.851135 25161 raft_consensus.cc:729] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb1bdadd0fb447edb5ab4013d86ceaad, State: Initialized, Role: FOLLOWER
I20260809 02:17:12.851368 25161 consensus_queue.cc:260] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } }
I20260809 02:17:12.851778 25161 ts_tablet_manager.cc:1428] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:12.852802 25276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.839068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:55089)
0809 02:17:12.839386 (+ 318us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.839430 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.840033 (+ 603us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.840997 (+ 964us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.841001 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.841391 (+ 390us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:12.841669 (+ 278us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.841676 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.843438 (+ 1762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.843439 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.844722 (+ 1283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.844753 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.845720 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.845721 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.845857 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.846479 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:12.846490 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:12.848456 (+ 1966us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:12.850872 (+ 2416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.850876 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.850878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.851176 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.851490 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:12.851491 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:12.851492 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:12.851628 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:12.852118 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:12.852122 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:12.852237 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.852366 (+ 129us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.852581 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":103,"threads_started":1}
I20260809 02:17:12.852813 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.839200 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:55089 (local address 127.9.40.126:43661)
0809 02:17:12.840399 (+ 1199us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.840408 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.840449 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:17:12.840584 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.840591 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.840647 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.840855 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:12.841820 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.843295 (+ 1475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.844865 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.845475 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.845564 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.848594 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:12.848599 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:12.848600 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:12.848612 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:12.850771 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.851279 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.851283 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.851283 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.851403 (+ 120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:12.851716 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:12.851717 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:12.851717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:12.852025 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:12.852075 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.852425 (+ 350us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.852568 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1142,"thread_start_us":106,"threads_started":1}
I20260809 02:17:12.853717 25274 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:12.853817 25274 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:12.854029 25274 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:12.854837 25071 ts_manager.cc:194] Registered new tserver with Master: eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385)
I20260809 02:17:12.855933 25071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:55089
I20260809 02:17:12.856931 25274 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
I20260809 02:17:12.860668 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.856228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46780 (local address 127.9.40.65:33385)
0809 02:17:12.856384 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:12.856388 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:12.857181 (+ 793us) server_negotiation.cc:407] Connection header received
0809 02:17:12.857384 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:12.857387 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:12.857401 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:12.857471 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:12.858061 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.858712 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.859780 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.860238 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.860327 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.860384 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:12.860555 (+ 171us) server_negotiation.cc:299] Negotiation successful
0809 02:17:12.860593 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260809 02:17:12.860667 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:12.855439 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.0.0.1:46780)
0809 02:17:12.856929 (+ 1490us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:12.856967 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:12.857210 (+ 243us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:12.857489 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:12.857492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:12.857494 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:12.857983 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.857986 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.858808 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.858809 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.859673 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:12.859676 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:12.860327 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:12.860328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:12.860436 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:12.860456 (+ 20us) client_negotiation.cc:769] Sending connection context
0809 02:17:12.860539 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:17:12.860594 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"thread_start_us":97,"threads_started":1}
I20260809 02:17:12.861295 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet 4dcc26b96af9423fabaf7113f33a84e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:12 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:46780
I20260809 02:17:12.861774 25283 tablet_replica.cc:330] stopping tablet replica
I20260809 02:17:12.861903 25283 raft_consensus.cc:2229] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:12.861972 25283 raft_consensus.cc:2258] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:12.862282 25283 ts_tablet_manager.cc:1905] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:12.864286 25283 ts_tablet_manager.cc:1918] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:12.864373 25283 log.cc:1196] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/4dcc26b96af9423fabaf7113f33a84e0
I20260809 02:17:12.864661 25283 ts_tablet_manager.cc:1939] T 4dcc26b96af9423fabaf7113f33a84e0 P eb1bdadd0fb447edb5ab4013d86ceaad: Deleting consensus metadata
I20260809 02:17:12.865232 25055 catalog_manager.cc:4841] TS eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385): tablet 4dcc26b96af9423fabaf7113f33a84e0 (table test-table [id=e46d807d353b4f91b025e96598774f6f]) successfully deleted
W20260809 02:17:12.935753 25279 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:12.935984 25279 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:12.936010 25279 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:12.937316 25279 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:12.937364 25279 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:12.937392 25279 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:12.938920 25279 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:39129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=41463
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25279
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:12.939200 25279 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:12.939461 25279 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:12.944854 25292 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.944900 25279 server_base.cc:1034] running on GCE node
W20260809 02:17:12.944870 25290 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:12.944902 25289 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:12.945325 25279 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:12.945536 25279 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:12.946712 25279 hybrid_clock.cc:648] HybridClock initialized: now 1786241832946681 us; error 41 us; skew 500 ppm
Aug 09 02:17:12 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:12.949669 25279 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:12.950158 25279 webserver.cc:460] Webserver started at http://127.9.40.66:41463/ using document root <none> and password file <none>
I20260809 02:17:12.950402 25279 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:12.950492 25279 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:12.952003 25279 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:17:12.952808 25301 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.953035 25279 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:12.953164 25279 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "2e343b58bf74430fa8ec205171ff9d5d"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "99d1ebcd25a46906e4631cbc4a074349"
server_key_iv: "b38f7cf7f4ed7fb987c717670ddf678d"
server_key_version: "encryptionkey@0"
I20260809 02:17:12.953576 25279 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:12.994458 25279 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:12.997177 25279 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:12.997359 25279 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:12.997645 25279 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:12.997993 25279 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:12.998040 25279 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:12.998100 25279 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:12.998129 25279 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.004370 25279 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39129
I20260809 02:17:13.004405 25417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39129 every 8 connection(s)
I20260809 02:17:13.004876 25279 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:13.009346 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25279
I20260809 02:17:13.009639 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:35139
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=35027
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241832, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:13.016721 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.005728 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:50645 (local address 127.9.40.126:43661)
0809 02:17:13.005822 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.005824 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.006004 (+ 180us) server_negotiation.cc:407] Connection header received
0809 02:17:13.006597 (+ 593us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.006598 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.006613 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.006682 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:13.007664 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.008235 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.009516 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.009769 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.009839 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.012885 (+ 3046us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:13.012888 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:13.012888 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:13.012897 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:13.014899 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.015399 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.015401 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.015402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.015473 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.015780 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.015781 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.015782 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.015962 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:13.016056 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.016458 (+ 402us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.016573 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260809 02:17:13.016839 25420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.005687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:50645)
0809 02:17:13.005870 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.005909 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.006439 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.006740 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.006744 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.007350 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:13.007562 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.007567 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.008330 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.008330 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.009369 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.009372 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.009921 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.009922 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.010086 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.010712 (+ 626us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:13.010723 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:13.012720 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:13.015013 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:13.015022 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:13.015025 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:13.015270 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:13.015564 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:13.015565 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:13.015565 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:13.015699 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:13.016147 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:13.016155 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:13.016299 (+ 144us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.016407 (+ 108us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.016558 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":111,"threads_started":1}
I20260809 02:17:13.017439 25418 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:13.017562 25418 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:13.017776 25418 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:13.018348 25071 ts_manager.cc:194] Registered new tserver with Master: 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129)
I20260809 02:17:13.018918 25071 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:50645
W20260809 02:17:13.101531 25421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:13.101754 25421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:13.101783 25421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:13.102948 25421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:13.102985 25421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:13.103011 25421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:13.104427 25421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:35139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=35027
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:13.104620 25421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:13.104843 25421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:13.110977 25427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.110994 25426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.111065 25430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.111387 25421 server_base.cc:1034] running on GCE node
I20260809 02:17:13.111657 25421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:13.112134 25421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:13.113348 25421 hybrid_clock.cc:648] HybridClock initialized: now 1786241833113322 us; error 42 us; skew 500 ppm
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:13.116291 25421 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:13.116937 25421 webserver.cc:460] Webserver started at http://127.9.40.67:35027/ using document root <none> and password file <none>
I20260809 02:17:13.117151 25421 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:13.117211 25421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:13.118716 25421 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:13.119463 25437 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.119850 25421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:13.119995 25421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "6ad2724437bb43c0a3f673d0182202c7"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "3b3c81ad684fd88682d279819930de6d"
server_key_iv: "54bff072fd0eb63c56b4b19b3219e251"
server_key_version: "encryptionkey@0"
I20260809 02:17:13.120527 25421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:13.133985 25421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:13.136065 25421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:13.136227 25421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:13.136482 25421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:13.136862 25421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:13.136900 25421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.136922 25421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:13.136935 25421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.143388 25421 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35139
I20260809 02:17:13.143432 25558 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35139 every 8 connection(s)
I20260809 02:17:13.143784 25421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:13.147428 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25421
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:13.157424 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.144556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53565 (local address 127.9.40.126:43661)
0809 02:17:13.144627 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.144630 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.144860 (+ 230us) server_negotiation.cc:407] Connection header received
0809 02:17:13.145404 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.145405 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.145418 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.145477 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:13.146469 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.146975 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.148225 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.148495 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.148549 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.153442 (+ 4893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:13.153443 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:13.153443 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:13.153452 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:13.155611 (+ 2159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.156219 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.156220 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.156221 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.156279 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.156566 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.156567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.156568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.156785 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:13.156877 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.157234 (+ 357us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.157330 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:17:13.157635 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.144537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:53565)
0809 02:17:13.144735 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.144772 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.145291 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.145545 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.145549 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.146154 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:13.146359 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.146363 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.147085 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.147086 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.148130 (+ 1044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.148133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.148583 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.148584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.148786 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.149961 (+ 1175us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:13.149971 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:13.153302 (+ 3331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:13.155697 (+ 2395us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:13.155706 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:13.155709 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:13.156079 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:13.156364 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:13.156365 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:13.156365 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:13.156482 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:13.156867 (+ 385us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:13.156874 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:13.157071 (+ 197us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.157197 (+ 126us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.157367 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":98,"threads_started":1}
I20260809 02:17:13.157630 25564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.147898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35142 (local address 127.9.40.126:43661)
0809 02:17:13.148014 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.148016 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.148140 (+ 124us) server_negotiation.cc:407] Connection header received
0809 02:17:13.148219 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.148220 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.148232 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.148305 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:13.149115 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.149566 (+ 451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.150733 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.152120 (+ 1387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.152284 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.154180 (+ 1896us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:13.154182 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:13.154183 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:13.154192 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:13.156021 (+ 1829us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.156483 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.156484 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.156485 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.156543 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:13.156853 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:13.156854 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:13.156855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:13.156987 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:13.157197 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.157410 (+ 213us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.157506 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":96,"thread_start_us":72,"threads_started":1}
I20260809 02:17:13.158183 25559 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43661
I20260809 02:17:13.158299 25559 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:13.158509 25559 heartbeater.cc:511] Master 127.9.40.126:43661 requested a full tablet report, sending...
I20260809 02:17:13.158871 25070 ts_manager.cc:194] Registered new tserver with Master: 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
I20260809 02:17:13.159284 25070 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:53565
I20260809 02:17:13.169795 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:13.172811 25277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.170261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35156 (local address 127.9.40.126:43661)
0809 02:17:13.170307 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.170309 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.170434 (+ 125us) server_negotiation.cc:407] Connection header received
0809 02:17:13.170513 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.170514 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.170556 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.170626 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:13.170744 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.171230 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.172032 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.172243 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.172296 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.172437 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:13.172536 (+ 99us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:13.172595 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.172651 (+ 56us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.172666 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:17:13.173161 25070 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:35156:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:13.174820 25070 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:13.177551 25202 tablet_service.cc:1467] Processing CreateTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 (DEFAULT_TABLE table=test-table [id=9a369bfd0a4f4ef6886ed205d15530ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:13.177783 25202 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a276487161e4c85984d9c00c1feb9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:13.179363 25161 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap starting.
I20260809 02:17:13.179692 25161 tablet_bootstrap.cc:654] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:13.180289 25161 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: No bootstrap required, opened a new log
I20260809 02:17:13.180361 25161 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.180478 25161 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.180562 25161 raft_consensus.cc:374] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:13.180608 25161 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb1bdadd0fb447edb5ab4013d86ceaad, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.180665 25161 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.180781 25161 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260809 02:17:13.180819 25281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.176922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39129 (local address 127.0.0.1:33792)
0809 02:17:13.176985 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.176996 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.177036 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.177432 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.177433 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.177436 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:13.177481 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.177483 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.178607 (+ 1124us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.178608 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.179492 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.179494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.180539 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.180541 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.180603 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.180608 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.180674 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.180721 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 02:17:13.180851 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.177052 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33792 (local address 127.9.40.66:39129)
0809 02:17:13.177247 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.177252 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.177263 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:13.177306 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.177309 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.177326 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.177559 (+ 233us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:13.177641 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.178497 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.179576 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.180451 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.180523 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.180579 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.180689 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.180757 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260809 02:17:13.181385 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.177298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34418 (local address 127.9.40.67:35139)
0809 02:17:13.177493 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.177498 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.177508 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:17:13.177555 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.177557 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.177573 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.177653 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:13.177986 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.178979 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.180170 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.180918 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.180990 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.181043 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.181247 (+ 204us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.181298 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161,"thread_start_us":112,"threads_started":1}
I20260809 02:17:13.181591 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.177046 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35139 (local address 127.0.0.1:34418)
0809 02:17:13.177154 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.177166 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.177219 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.177768 (+ 549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.177769 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.177771 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:13.177819 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.177820 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.179098 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.179099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.180099 (+ 1000us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.180103 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.181082 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.181083 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.181148 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.181151 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.181470 (+ 319us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.181529 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":92,"thread_start_us":60,"threads_started":1}
I20260809 02:17:13.181711 25351 tablet_service.cc:1467] Processing CreateTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 (DEFAULT_TABLE table=test-table [id=9a369bfd0a4f4ef6886ed205d15530ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:13.182099 25351 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a276487161e4c85984d9c00c1feb9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:13.182231 25490 tablet_service.cc:1467] Processing CreateTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 (DEFAULT_TABLE table=test-table [id=9a369bfd0a4f4ef6886ed205d15530ab]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:13.182550 25490 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a276487161e4c85984d9c00c1feb9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:13.185469 25570 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Bootstrap starting.
I20260809 02:17:13.185503 25571 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Bootstrap starting.
I20260809 02:17:13.186200 25570 tablet_bootstrap.cc:654] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:13.186280 25571 tablet_bootstrap.cc:654] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:13.186708 25570 log.cc:824] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:13.186728 25571 log.cc:824] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:13.187439 25570 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: No bootstrap required, opened a new log
I20260809 02:17:13.187436 25571 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: No bootstrap required, opened a new log
I20260809 02:17:13.187520 25571 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:17:13.187523 25570 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:13.189054 25571 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.189236 25571 raft_consensus.cc:374] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:13.189284 25571 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e343b58bf74430fa8ec205171ff9d5d, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.189182 25570 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.189338 25570 raft_consensus.cc:374] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:13.189396 25570 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad2724437bb43c0a3f673d0182202c7, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.189429 25571 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.189518 25570 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.189723 25571 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:13.189803 25570 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
I20260809 02:17:13.189862 25418 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
I20260809 02:17:13.189904 25559 heartbeater.cc:503] Master 127.9.40.126:43661 was elected leader, sending a full tablet report...
I20260809 02:17:13.221916 25575 raft_consensus.cc:482] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:13.222055 25575 raft_consensus.cc:504] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.222473 25575 leader_election.cc:288] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129), eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385)
I20260809 02:17:13.225749 25577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.222769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.9.40.67:55123)
0809 02:17:13.222998 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.223006 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.223096 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.223247 (+ 151us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.223248 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.223250 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:13.223329 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.223330 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.224039 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.224040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.224885 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.224888 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.225461 (+ 573us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.225462 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.225536 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.225540 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.225616 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.225664 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":103,"threads_started":1}
I20260809 02:17:13.225742 25282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.222877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:55123 (local address 127.9.40.65:33385)
0809 02:17:13.222931 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.222933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.223074 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:17:13.223159 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.223160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.223171 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.223244 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:13.223410 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.223932 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.224976 (+ 1044us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.225369 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.225457 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.225468 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.225622 (+ 154us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.225667 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:17:13.225935 25562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.222653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39129 (local address 127.9.40.67:36843)
0809 02:17:13.222771 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.222786 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.222852 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.223081 (+ 229us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:13.223082 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:13.223084 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:13.223170 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.223172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.223894 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.223895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.224894 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:13.224897 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.225596 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.225597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:13.225699 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.225707 (+ 8us) client_negotiation.cc:769] Sending connection context
0809 02:17:13.225781 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:17:13.225827 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":87,"mutex_wait_us":11}
I20260809 02:17:13.226276 25567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.222845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36843 (local address 127.9.40.66:39129)
0809 02:17:13.222901 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.222903 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.222914 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:13.222958 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.222959 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.222971 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.223074 (+ 103us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:13.223260 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.223794 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.225062 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.225502 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.225597 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.225609 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.225789 (+ 180us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.226223 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:17:13.226401 25227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "6ad2724437bb43c0a3f673d0182202c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" is_pre_election: true
I20260809 02:17:13.226579 25227 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad2724437bb43c0a3f673d0182202c7 in term 0.
I20260809 02:17:13.226707 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "6ad2724437bb43c0a3f673d0182202c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e343b58bf74430fa8ec205171ff9d5d" is_pre_election: true
I20260809 02:17:13.226894 25371 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6ad2724437bb43c0a3f673d0182202c7 in term 0.
I20260809 02:17:13.226861 25441 leader_election.cc:302] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6ad2724437bb43c0a3f673d0182202c7, eb1bdadd0fb447edb5ab4013d86ceaad; no voters:
I20260809 02:17:13.227037 25575 raft_consensus.cc:2781] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:13.227108 25575 raft_consensus.cc:482] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:13.227144 25575 raft_consensus.cc:3037] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:13.228013 25575 raft_consensus.cc:504] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.228145 25575 leader_election.cc:288] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [CANDIDATE]: Term 1 election: Requested vote from peers 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129), eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385)
I20260809 02:17:13.228377 25371 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "6ad2724437bb43c0a3f673d0182202c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2e343b58bf74430fa8ec205171ff9d5d"
I20260809 02:17:13.228479 25371 raft_consensus.cc:3037] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:13.228528 25227 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "6ad2724437bb43c0a3f673d0182202c7" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
I20260809 02:17:13.228634 25227 raft_consensus.cc:3037] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:13.229435 25371 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad2724437bb43c0a3f673d0182202c7 in term 1.
I20260809 02:17:13.229573 25227 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6ad2724437bb43c0a3f673d0182202c7 in term 1.
I20260809 02:17:13.229705 25441 leader_election.cc:302] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e343b58bf74430fa8ec205171ff9d5d, 6ad2724437bb43c0a3f673d0182202c7; no voters:
I20260809 02:17:13.229835 25575 raft_consensus.cc:2781] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:13.230010 25575 raft_consensus.cc:686] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 LEADER]: Becoming Leader. State: Replica: 6ad2724437bb43c0a3f673d0182202c7, State: Running, Role: LEADER
I20260809 02:17:13.230160 25575 consensus_queue.cc:237] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.231204 25071 catalog_manager.cc:5495] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 reported cstate change: term changed from 0 to 1, leader changed from <none> to 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "6ad2724437bb43c0a3f673d0182202c7" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } health_report { overall_health: UNKNOWN } } }
I20260809 02:17:13.244397 25568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.241686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34432 (local address 127.9.40.67:35139)
0809 02:17:13.241743 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:13.241745 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:13.241781 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:17:13.241826 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:13.241827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:13.241846 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:13.241900 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:13.242156 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.242622 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.243612 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:13.243792 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:13.243851 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:13.244020 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:13.244122 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:13.244188 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:13.244293 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:17:13.244311 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260809 02:17:13.246371 25371 raft_consensus.cc:1261] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Refusing update from remote peer 6ad2724437bb43c0a3f673d0182202c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:13.246411 25227 raft_consensus.cc:1261] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Refusing update from remote peer 6ad2724437bb43c0a3f673d0182202c7: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:13.246825 25575 consensus_queue.cc:1035] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:13.247035 25578 consensus_queue.cc:1035] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:13.250227 25583 mvcc.cc:205] Tried to move back new op lower bound from 7316446548974211072 to 7316446548911673344. Current Snapshot: MvccSnapshot[applied={T|T < 7316446548974211072}]
I20260809 02:17:13.251748 25585 mvcc.cc:205] Tried to move back new op lower bound from 7316446548974211072 to 7316446548911673344. Current Snapshot: MvccSnapshot[applied={T|T < 7316446548974211072}]
I20260809 02:17:13.251794 25070 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:13.251914 25070 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:35156:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:13.252817 25070 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Sending DeleteTablet for 3 replicas of tablet 8a276487161e4c85984d9c00c1feb9e2
I20260809 02:17:13.253294 25202 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:13 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:46780
I20260809 02:17:13.253331 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25112
I20260809 02:17:13.253347 25351 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:13 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:33792
I20260809 02:17:13.253470 25490 tablet_service.cc:1514] Processing DeleteTablet for tablet 8a276487161e4c85984d9c00c1feb9e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:13 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:34418
I20260809 02:17:13.253652 25283 tablet_replica.cc:330] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: stopping tablet replica
I20260809 02:17:13.253710 25283 raft_consensus.cc:2229] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:13.255228 25592 tablet_replica.cc:330] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: stopping tablet replica
I20260809 02:17:13.255458 25592 raft_consensus.cc:2229] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:13.255573 25592 raft_consensus.cc:2258] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:13.255738 25593 tablet_replica.cc:330] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: stopping tablet replica
I20260809 02:17:13.255829 25593 raft_consensus.cc:2229] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:13.255923 25593 raft_consensus.cc:2258] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:13.256373 25593 ts_tablet_manager.cc:1905] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:13.258602 25593 ts_tablet_manager.cc:1918] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:13.258680 25593 log.cc:1196] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/8a276487161e4c85984d9c00c1feb9e2
I20260809 02:17:13.259014 25593 ts_tablet_manager.cc:1939] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Deleting consensus metadata
I20260809 02:17:13.261049 25055 catalog_manager.cc:4841] TS 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139): tablet 8a276487161e4c85984d9c00c1feb9e2 (table test-table [id=9a369bfd0a4f4ef6886ed205d15530ab]) successfully deleted
I20260809 02:17:13.261559 25592 ts_tablet_manager.cc:1905] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:17:13.262171 25055 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:33385 (error 108)
I20260809 02:17:13.262625 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25279
I20260809 02:17:13.262776 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.262617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.0.0.1:46784)
0809 02:17:13.262686 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.262732 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260809 02:17:13.264322 25055 catalog_manager.cc:4584] TS eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8a276487161e4c85984d9c00c1feb9e2: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33385: connect: Connection refused (error 111)
I20260809 02:17:13.269007 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25421
I20260809 02:17:13.269111 25566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.268977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39129 (local address 127.0.0.1:33796)
0809 02:17:13.269039 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.269074 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:39129: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:17:13.275254 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25036
I20260809 02:17:13.282651 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--webserver_interface=127.9.40.126
--webserver_port=34853
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43661
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:13.367678 25595 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:13.367983 25595 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:13.368028 25595 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:13.369329 25595 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:13.369408 25595 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:13.369436 25595 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:13.369460 25595 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:13.369493 25595 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:13.371235 25595 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43661
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43661
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=34853
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25595
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:13.371562 25595 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:13.371821 25595 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:13.377619 25601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.377799 25600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.377635 25603 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.377899 25595 server_base.cc:1034] running on GCE node
I20260809 02:17:13.378144 25595 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:13.378417 25595 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:13.379583 25595 hybrid_clock.cc:648] HybridClock initialized: now 1786241833379562 us; error 36 us; skew 500 ppm
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:13.382295 25595 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:13.382805 25595 webserver.cc:460] Webserver started at http://127.9.40.126:34853/ using document root <none> and password file <none>
I20260809 02:17:13.383049 25595 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:13.383119 25595 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:13.384450 25595 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.385177 25610 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.385381 25595 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.385550 25595 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "5972ac8f885d44e391823127102fc8fd"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "9d9256e664e78fb39f49f4c44c662079"
server_key_iv: "5ad6fedeb5e05454e5549d0b831dbef4"
server_key_version: "encryptionkey@0"
I20260809 02:17:13.385927 25595 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:13.401659 25595 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:13.403700 25595 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:13.403877 25595 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:13.408351 25595 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43661
I20260809 02:17:13.408388 25664 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43661 every 8 connection(s)
I20260809 02:17:13.408824 25595 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:13.410452 25665 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:17:13.411135 25665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap starting.
I20260809 02:17:13.418556 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25595
I20260809 02:17:13.418921 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:33385
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=36937
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:13.420409 25665 log.cc:824] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:13.423406 25665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:13.423794 25665 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd: Bootstrap complete.
I20260809 02:17:13.428046 25665 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:13.428459 25665 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.428659 25665 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:13.428783 25665 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:13.428850 25665 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:13.428927 25665 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:17:13.430634 25665 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:13.430765 25665 leader_election.cc:302] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5972ac8f885d44e391823127102fc8fd; no voters:
I20260809 02:17:13.430990 25665 leader_election.cc:288] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:17:13.431067 25669 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:17:13.431231 25669 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [term 3 LEADER]: Becoming Leader. State: Replica: 5972ac8f885d44e391823127102fc8fd, State: Running, Role: LEADER
I20260809 02:17:13.431300 25665 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:13.431346 25669 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } }
I20260809 02:17:13.431619 25670 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:13.431699 25670 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:13.432009 25671 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5972ac8f885d44e391823127102fc8fd. Latest consensus state: current_term: 3 leader_uuid: "5972ac8f885d44e391823127102fc8fd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5972ac8f885d44e391823127102fc8fd" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43661 } } }
I20260809 02:17:13.432171 25671 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5972ac8f885d44e391823127102fc8fd [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:13.432300 25673 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:13.433221 25673 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:13.433344 25673 catalog_manager.cc:1259] Loaded cluster ID: 00bafdd6337e4ef0b7662ff57e004887
I20260809 02:17:13.433594 25673 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260809 02:17:13.433992 25673 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9f1396b9ad google::LogMessage::Fail() at ??:0
@ 0x7f9f1396e88a google::LogMessage::SendToLog() at ??:0
@ 0x7f9f1396b4de google::LogMessage::Flush() at ??:0
@ 0x7f9f1396ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f9f1b88c0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f9f1b8958b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9f158390d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9f1582e954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f9f166926db start_thread at ??:0
@ 0x7f9f1246961f clone at ??:0
W20260809 02:17:13.522176 25668 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:13.522418 25668 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:13.522457 25668 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:13.523592 25668 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:13.523656 25668 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:13.523701 25668 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:13.525269 25668 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:33385
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=36937
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25668
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:13.525558 25668 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:13.525853 25668 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:13.531103 25704 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.531162 25701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.531221 25668 server_base.cc:1034] running on GCE node
W20260809 02:17:13.531376 25702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.531646 25668 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:13.531953 25668 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:13.533135 25668 hybrid_clock.cc:648] HybridClock initialized: now 1786241833533116 us; error 26 us; skew 500 ppm
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:13.536362 25668 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:13.536957 25668 webserver.cc:460] Webserver started at http://127.9.40.65:36937/ using document root <none> and password file <none>
I20260809 02:17:13.537220 25668 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:13.537268 25668 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:13.538637 25668 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.539395 25714 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.539587 25668 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.539677 25668 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "eb1bdadd0fb447edb5ab4013d86ceaad"
format_stamp: "Formatted at 2026-08-09 02:17:11 on dist-test-slave-qg60"
server_key: "ea164fa018aeddd570668d52a390c852"
server_key_iv: "4347d5404545297152878438c444fae0"
server_key_version: "encryptionkey@0"
I20260809 02:17:13.540053 25668 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:13.553997 25668 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:13.556440 25668 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:13.556602 25668 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:13.556890 25668 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:13.557391 25722 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:17:13.559541 25668 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:17:13.559625 25668 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:17:13.559675 25668 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:17:13.560350 25668 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:17:13.560408 25668 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:13.560479 25722 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap starting.
I20260809 02:17:13.568023 25668 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33385
I20260809 02:17:13.568156 25837 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33385 every 8 connection(s)
I20260809 02:17:13.568485 25668 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:13.572862 25722 log.cc:824] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:13.575734 25722 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:13.576102 25722 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Bootstrap complete.
I20260809 02:17:13.576613 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25668
I20260809 02:17:13.576951 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:39129
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=41463
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:13.577561 25722 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent bootstrapping tablet: real 0.017s user 0.008s sys 0.007s
I20260809 02:17:13.579190 25722 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.579579 25722 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: eb1bdadd0fb447edb5ab4013d86ceaad, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.579761 25722 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.580084 25722 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
W20260809 02:17:13.639179 25840 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:17:13.569309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:45437)
0809 02:17:13.569528 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.569568 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:13.570093 (+ 525us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:13.638945 (+ 68852us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":123,"threads_started":1}
W20260809 02:17:13.639478 25838 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:43661: Client connection negotiation failed: client connection to 127.9.40.126:43661: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:17:13.639837 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.639711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:38393)
0809 02:17:13.639768 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.639806 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:17:13.640208 25840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.640123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:33639)
0809 02:17:13.640169 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.640192 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:17:13.640337 25838 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:17:13.664328 25842 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:13.664543 25842 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:13.664569 25842 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:13.665822 25842 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:13.665866 25842 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:13.665894 25842 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:13.667418 25842 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:39129
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=41463
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25842
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:13.667680 25842 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:13.667927 25842 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:13.673025 25853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.673146 25851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.673255 25850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.673269 25842 server_base.cc:1034] running on GCE node
I20260809 02:17:13.673583 25842 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:13.673780 25842 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:13.674983 25842 hybrid_clock.cc:648] HybridClock initialized: now 1786241833674965 us; error 34 us; skew 500 ppm
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:13.677577 25842 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:13.678086 25842 webserver.cc:460] Webserver started at http://127.9.40.66:41463/ using document root <none> and password file <none>
I20260809 02:17:13.678319 25842 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:13.678390 25842 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:13.679804 25842 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:13.680613 25861 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.680858 25842 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:13.681136 25842 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "2e343b58bf74430fa8ec205171ff9d5d"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "99d1ebcd25a46906e4631cbc4a074349"
server_key_iv: "b38f7cf7f4ed7fb987c717670ddf678d"
server_key_version: "encryptionkey@0"
I20260809 02:17:13.681526 25842 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:13.697038 25842 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:13.699177 25842 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:13.699368 25842 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:13.699627 25842 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:13.700130 25868 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:17:13.701761 25842 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:17:13.701825 25842 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:17:13.701872 25842 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:17:13.702559 25842 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:17:13.702615 25842 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:13.702713 25868 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Bootstrap starting.
I20260809 02:17:13.704430 25868 log.cc:824] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:13.706920 25868 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:13.707247 25868 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Bootstrap complete.
I20260809 02:17:13.707436 25868 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260809 02:17:13.708873 25868 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.709156 25868 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e343b58bf74430fa8ec205171ff9d5d, State: Initialized, Role: FOLLOWER
I20260809 02:17:13.709359 25868 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:13.709722 25868 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260809 02:17:13.710297 25842 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39129
I20260809 02:17:13.710348 25986 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39129 every 8 connection(s)
I20260809 02:17:13.710651 25842 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:13.711635 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.711264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:32777)
0809 02:17:13.711442 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.711517 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":67,"threads_started":1}
W20260809 02:17:13.711843 25987 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:43661: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
I20260809 02:17:13.712154 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.712054 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:40995)
0809 02:17:13.712107 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.712133 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:17:13.712440 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.712354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:46391)
0809 02:17:13.712398 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.712420 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 02:17:13.712553 25987 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:17:13.713647 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25842
I20260809 02:17:13.713901 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:35139
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:43661
--webserver_port=35027
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:33937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:13.798301 25990 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:13.798538 25990 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:13.798565 25990 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:13.799834 25990 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:13.799883 25990 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:13.799911 25990 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:13.801405 25990 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:33937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:35139
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=35027
--tserver_master_addrs=127.9.40.126:43661
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25990
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:13.801604 25990 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:13.801797 25990 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:13.807214 25999 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.807247 25990 server_base.cc:1034] running on GCE node
W20260809 02:17:13.807274 25995 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:13.807242 25996 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:13.807739 25990 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:13.808010 25990 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:13.809156 25990 hybrid_clock.cc:648] HybridClock initialized: now 1786241833809141 us; error 33 us; skew 500 ppm
Aug 09 02:17:13 dist-test-slave-qg60 krb5kdc[24443](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:13.811740 25990 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:13.812245 25990 webserver.cc:460] Webserver started at http://127.9.40.67:35027/ using document root <none> and password file <none>
I20260809 02:17:13.812486 25990 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:13.812557 25990 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:13.814028 25990 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.814705 26006 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.814889 25990 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:13.814996 25990 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "6ad2724437bb43c0a3f673d0182202c7"
format_stamp: "Formatted at 2026-08-09 02:17:12 on dist-test-slave-qg60"
server_key: "3b3c81ad684fd88682d279819930de6d"
server_key_iv: "54bff072fd0eb63c56b4b19b3219e251"
server_key_version: "encryptionkey@0"
I20260809 02:17:13.815371 25990 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:13.839468 25990 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:13.841876 25990 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:13.842048 25990 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:13.842334 25990 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:13.842672 25990 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:13.842728 25990 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.842772 25990 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:13.842801 25990 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:13.849200 25990 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:35139
I20260809 02:17:13.849260 26128 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:35139 every 8 connection(s)
I20260809 02:17:13.849614 25990 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:13.850004 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 25990
I20260809 02:17:13.850865 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.850390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:43315)
0809 02:17:13.850583 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.850701 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":97,"threads_started":1}
W20260809 02:17:13.851294 26129 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:43661 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:43661: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
I20260809 02:17:13.851573 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.851497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:58813)
0809 02:17:13.851528 (+ 31us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.851556 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260809 02:17:13.851894 26132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:13.851802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:54605)
0809 02:17:13.851848 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:13.851874 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:17:13.852065 26129 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:17:14.641011 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:14.640766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:57275)
0809 02:17:14.640899 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:14.640939 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":65,"threads_started":1}
I20260809 02:17:14.713150 26134 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:14.712933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:45527)
0809 02:17:14.713049 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:14.713085 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":57,"threads_started":1}
I20260809 02:17:14.852626 26135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:14.852412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:55021)
0809 02:17:14.852532 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:14.852572 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":58,"threads_started":1}
I20260809 02:17:15.052860 26136 raft_consensus.cc:482] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:15.053035 26136 raft_consensus.cc:504] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:15.053321 26136 leader_election.cc:288] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129), 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
Aug 09 02:17:15 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
Aug 09 02:17:15 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260809 02:17:15.065797 26138 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.053696 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:35139 (local address 127.9.40.65:58927)
0809 02:17:15.053828 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.053836 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:15.053873 (+ 37us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:15.054845 (+ 972us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:15.054846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:15.055022 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:15.055194 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:15.055202 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.056890 (+ 1688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.056891 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:15.057712 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:15.057714 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.058488 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.058489 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:15.058599 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:15.059257 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:15.059266 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:15.061531 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:15.063864 (+ 2333us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:15.063872 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:15.063874 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:15.064184 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:15.064462 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064463 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:15.064464 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:15.064586 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:15.065347 (+ 761us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:15.065349 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:15.065405 (+ 56us) client_negotiation.cc:769] Sending connection context
0809 02:17:15.065483 (+ 78us) client_negotiation.cc:240] Negotiation successful
0809 02:17:15.065687 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":112,"mutex_wait_us":34,"thread_start_us":65,"threads_started":1}
I20260809 02:17:15.065810 26139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.053789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58927 (local address 127.9.40.67:35139)
0809 02:17:15.053954 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:15.053960 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:15.053986 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:17:15.054621 (+ 635us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:15.054624 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:15.054682 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:15.054832 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:15.055332 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.056756 (+ 1424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.057814 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.058359 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.058518 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:15.061646 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:15.061650 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:15.061650 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:15.061670 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:15.063743 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064275 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:15.064278 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:15.064279 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:15.064375 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064664 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:15.064665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:15.064666 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:15.065273 (+ 607us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:15.065351 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:15.065563 (+ 212us) server_negotiation.cc:299] Negotiation successful
0809 02:17:15.065692 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":73,"threads_started":1}
I20260809 02:17:15.065871 26137 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.053671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:47061 (local address 127.9.40.66:39129)
0809 02:17:15.053863 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:15.053876 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:15.053902 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:17:15.054042 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:15.054045 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:15.054107 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:15.054283 (+ 176us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:15.055326 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.056757 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.057951 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.058350 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.058464 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:15.061251 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:15.061256 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:15.061257 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:15.061272 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:15.063773 (+ 2501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064285 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:15.064289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:15.064289 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:15.064388 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064677 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:15.064678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:15.064678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:15.065086 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:15.065186 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:15.065551 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:17:15.065727 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":58,"threads_started":1}
I20260809 02:17:15.066087 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.053556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39129 (local address 127.9.40.65:47061)
0809 02:17:15.053636 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.053653 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:15.053741 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:15.054371 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:15.054375 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:15.054827 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:15.055194 (+ 367us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:15.055208 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.056864 (+ 1656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.056865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:15.057846 (+ 981us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:15.057849 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:15.058458 (+ 609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:15.058460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:15.058599 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:15.059305 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:15.059310 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:15.061113 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:15.063898 (+ 2785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:15.063899 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:15.063900 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:15.064190 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:15.064470 (+ 280us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:15.064471 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:15.064472 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:15.064590 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:15.065178 (+ 588us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:15.065185 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:15.065347 (+ 162us) client_negotiation.cc:769] Sending connection context
0809 02:17:15.065474 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:17:15.065776 (+ 302us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53,"mutex_wait_us":142}
I20260809 02:17:15.066570 25939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2e343b58bf74430fa8ec205171ff9d5d" is_pre_election: true
I20260809 02:17:15.066741 25939 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate eb1bdadd0fb447edb5ab4013d86ceaad in term 1.
W20260809 02:17:15.066980 25717 leader_election.cc:341] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139): Not found: Tablet not found: 8a276487161e4c85984d9c00c1feb9e2
I20260809 02:17:15.067186 25717 leader_election.cc:302] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e343b58bf74430fa8ec205171ff9d5d, eb1bdadd0fb447edb5ab4013d86ceaad; no voters: 6ad2724437bb43c0a3f673d0182202c7
I20260809 02:17:15.067360 26136 raft_consensus.cc:2781] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:17:15.067438 26136 raft_consensus.cc:482] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:15.067457 26136 raft_consensus.cc:3037] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:17:15.068621 26136 raft_consensus.cc:504] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:15.068854 26136 leader_election.cc:288] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 election: Requested vote from peers 2e343b58bf74430fa8ec205171ff9d5d (127.9.40.66:39129), 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139)
I20260809 02:17:15.069121 25939 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8a276487161e4c85984d9c00c1feb9e2" candidate_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "2e343b58bf74430fa8ec205171ff9d5d"
W20260809 02:17:15.069222 25717 leader_election.cc:341] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139): Not found: Tablet not found: 8a276487161e4c85984d9c00c1feb9e2
I20260809 02:17:15.069226 25939 raft_consensus.cc:3037] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:17:15.070353 25939 raft_consensus.cc:2454] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate eb1bdadd0fb447edb5ab4013d86ceaad in term 2.
I20260809 02:17:15.070528 25717 leader_election.cc:302] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 2e343b58bf74430fa8ec205171ff9d5d, eb1bdadd0fb447edb5ab4013d86ceaad; no voters: 6ad2724437bb43c0a3f673d0182202c7
I20260809 02:17:15.070670 26136 raft_consensus.cc:2781] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:17:15.070863 26136 raft_consensus.cc:686] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [term 2 LEADER]: Becoming Leader. State: Replica: eb1bdadd0fb447edb5ab4013d86ceaad, State: Running, Role: LEADER
I20260809 02:17:15.070963 26136 consensus_queue.cc:237] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:15.071139 26133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.071037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:38021)
0809 02:17:15.071080 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.071115 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 02:17:15.456473 25717 consensus_peers.cc:487] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad -> Peer 6ad2724437bb43c0a3f673d0182202c7 (127.9.40.67:35139): Couldn't send request to peer 6ad2724437bb43c0a3f673d0182202c7. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 8a276487161e4c85984d9c00c1feb9e2. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:17:15.576529 25939 raft_consensus.cc:1261] T 8a276487161e4c85984d9c00c1feb9e2 P 2e343b58bf74430fa8ec205171ff9d5d [term 2 FOLLOWER]: Refusing update from remote peer eb1bdadd0fb447edb5ab4013d86ceaad: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 02:17:15.576963 26136 consensus_queue.cc:1035] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad [LEADER]: Connected to new peer: Peer: permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 02:17:15.577950 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.577757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:43943)
0809 02:17:15.577842 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.577893 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70,"thread_start_us":47,"threads_started":1}
I20260809 02:17:15.578917 26145 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.578804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:57595)
0809 02:17:15.578856 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.578882 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:17:15.578940 26149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.578641 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:57957)
0809 02:17:15.578873 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.578908 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":53,"threads_started":1}
I20260809 02:17:15.853384 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:15.853104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:48555)
0809 02:17:15.853287 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:15.853338 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":89,"threads_started":1}
I20260809 02:17:16.066325 26151 ts_tablet_manager.cc:927] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Initiating tablet copy from peer eb1bdadd0fb447edb5ab4013d86ceaad (127.9.40.65:33385)
I20260809 02:17:16.066576 26151 tablet_copy_client.cc:316] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.65:33385
Aug 09 02:17:16 dist-test-slave-qg60 krb5kdc[24443](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241833, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:17:16.074934 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.066895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33385 (local address 127.9.40.67:47005)
0809 02:17:16.066981 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.067001 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:16.067082 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:16.067388 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:16.067393 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:16.067683 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:16.067755 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:16.067757 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:16.068769 (+ 1012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:16.068770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:16.069406 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:16.069409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:16.069857 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:16.069858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:16.069915 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:16.070385 (+ 470us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:16.070389 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:16.071832 (+ 1443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:16.073807 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:16.073813 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:16.073815 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:16.074003 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:16.074248 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:16.074248 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:16.074249 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:16.074302 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:16.074628 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:16.074633 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:16.074680 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:17:16.074744 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:17:16.074852 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 02:17:16.074913 26153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.066999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:47005 (local address 127.9.40.65:33385)
0809 02:17:16.067233 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:16.067237 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:16.067250 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:17:16.067276 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:16.067277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:16.067291 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:16.067371 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:16.067863 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:16.068652 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:16.069485 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:16.069786 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:16.069848 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:16.071943 (+ 2095us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:16.071954 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:16.071955 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:16.071963 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:16.073730 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:16.074104 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:16.074107 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:16.074107 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:16.074177 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:16.074383 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:16.074384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:16.074384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:16.074554 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:16.074624 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:16.074777 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:17:16.074840 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":198,"thread_start_us":143,"threads_started":1}
I20260809 02:17:16.075498 25808 tablet_copy_service.cc:140] P eb1bdadd0fb447edb5ab4013d86ceaad: Received BeginTabletCopySession request for tablet 8a276487161e4c85984d9c00c1feb9e2 from peer 6ad2724437bb43c0a3f673d0182202c7 ({username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:47005)
I20260809 02:17:16.075568 25808 tablet_copy_service.cc:161] P eb1bdadd0fb447edb5ab4013d86ceaad: Beginning new tablet copy session on tablet 8a276487161e4c85984d9c00c1feb9e2 from peer 6ad2724437bb43c0a3f673d0182202c7 at {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:47005: session id = 6ad2724437bb43c0a3f673d0182202c7-8a276487161e4c85984d9c00c1feb9e2
I20260809 02:17:16.076069 25808 tablet_copy_source_session.cc:205] T 8a276487161e4c85984d9c00c1feb9e2 P eb1bdadd0fb447edb5ab4013d86ceaad: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:17:16.077037 26151 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8a276487161e4c85984d9c00c1feb9e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:16.080173 26151 tablet_copy_client.cc:791] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: tablet copy: Starting download of 0 data blocks...
I20260809 02:17:16.080351 26151 tablet_copy_client.cc:655] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: tablet copy: Starting download of 1 WAL segments...
I20260809 02:17:16.081399 26151 tablet_copy_client.cc:531] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:17:16.082141 26151 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Bootstrap starting.
I20260809 02:17:16.083501 26151 log.cc:824] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:16.086572 26151 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:16.086831 26151 tablet_bootstrap.cc:492] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Bootstrap complete.
I20260809 02:17:16.086994 26151 ts_tablet_manager.cc:1397] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.002s
I20260809 02:17:16.088083 26151 raft_consensus.cc:348] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:16.088270 26151 raft_consensus.cc:729] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ad2724437bb43c0a3f673d0182202c7, State: Initialized, Role: FOLLOWER
I20260809 02:17:16.088429 26151 consensus_queue.cc:260] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2e343b58bf74430fa8ec205171ff9d5d" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39129 } } peers { permanent_uuid: "6ad2724437bb43c0a3f673d0182202c7" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 35139 } } peers { permanent_uuid: "eb1bdadd0fb447edb5ab4013d86ceaad" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33385 } }
I20260809 02:17:16.088779 26151 ts_tablet_manager.cc:1428] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
I20260809 02:17:16.089118 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.088987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:39003)
0809 02:17:16.089046 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.089078 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:17:16.089183 25808 tablet_copy_service.cc:342] P eb1bdadd0fb447edb5ab4013d86ceaad: Request end of tablet copy session 6ad2724437bb43c0a3f673d0182202c7-8a276487161e4c85984d9c00c1feb9e2 received from {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:47005
I20260809 02:17:16.089277 25808 tablet_copy_service.cc:433] P eb1bdadd0fb447edb5ab4013d86ceaad: ending tablet copy session 6ad2724437bb43c0a3f673d0182202c7-8a276487161e4c85984d9c00c1feb9e2 on tablet 8a276487161e4c85984d9c00c1feb9e2 with peer 6ad2724437bb43c0a3f673d0182202c7
I20260809 02:17:16.089535 26150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.089407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:56203)
0809 02:17:16.089476 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.089509 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260809 02:17:16.579569 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.579336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:55141)
0809 02:17:16.579502 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.579534 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":75,"threads_started":1}
I20260809 02:17:16.579567 26156 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.579333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:50049)
0809 02:17:16.579466 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.579509 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":74,"threads_started":1}
I20260809 02:17:16.610760 26157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.610616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:49675)
0809 02:17:16.610679 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.610716 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:17:16.610759 26081 raft_consensus.cc:1203] T 8a276487161e4c85984d9c00c1feb9e2 P 6ad2724437bb43c0a3f673d0182202c7 [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 02:17:16.611441 26161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:16.611221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:57715)
0809 02:17:16.611358 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:16.611395 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":92,"threads_started":1}
I20260809 02:17:17.580238 26162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:17.580031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:46531)
0809 02:17:17.580159 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:17.580191 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":70,"threads_started":1}
I20260809 02:17:17.611544 26164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:17.611292 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:47855)
0809 02:17:17.611435 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:17.611467 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 02:17:17.612013 26165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:17.611819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:49801)
0809 02:17:17.611939 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:17.611975 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":49,"threads_started":1}
I20260809 02:17:18.581053 26166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:18.580815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:38255)
0809 02:17:18.580948 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:18.580998 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":67,"threads_started":1}
I20260809 02:17:18.612433 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:18.612173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:50571)
0809 02:17:18.612340 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:18.612384 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":115,"threads_started":1}
I20260809 02:17:18.612783 26169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:18.612487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:34951)
0809 02:17:18.612645 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:18.612705 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":74,"threads_started":1}
I20260809 02:17:18.612893 26168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:18.612780 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:59771)
0809 02:17:18.612819 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:18.612863 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 02:17:19.581859 26172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:19.581637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:54731)
0809 02:17:19.581770 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:19.581811 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 02:17:19.613673 26173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:19.613438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:47485)
0809 02:17:19.613590 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:19.613625 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":64,"threads_started":1}
I20260809 02:17:19.613698 26174 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:19.613413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:49437)
0809 02:17:19.613585 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:19.613624 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":114,"threads_started":1}
I20260809 02:17:20.582630 26175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:20.582393 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:60299)
0809 02:17:20.582543 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:20.582579 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":92,"threads_started":1}
I20260809 02:17:20.614413 26177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:20.614190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:42933)
0809 02:17:20.614327 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:20.614363 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":83,"threads_started":1}
I20260809 02:17:20.614414 26176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:20.614186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:35619)
0809 02:17:20.614333 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:20.614370 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":62,"threads_started":1}
I20260809 02:17:21.583351 26178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:21.583091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:56647)
0809 02:17:21.583266 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:21.583303 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":105,"threads_started":1}
I20260809 02:17:21.615118 26180 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:21.614876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:57807)
0809 02:17:21.615023 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:21.615058 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":91,"threads_started":1}
I20260809 02:17:21.615134 26179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:21.614872 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:39429)
0809 02:17:21.615029 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:21.615071 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":70,"threads_started":1}
I20260809 02:17:22.584106 26181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:22.583898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:60705)
0809 02:17:22.584018 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:22.584055 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":63,"threads_started":1}
I20260809 02:17:22.615820 26182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:22.615552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:43263)
0809 02:17:22.615705 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:22.615752 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":81,"threads_started":1}
I20260809 02:17:22.615908 26183 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:22.615619 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:59371)
0809 02:17:22.615830 (+ 211us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:22.615866 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":192,"thread_start_us":97,"threads_started":1}
I20260809 02:17:23.584868 26186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:23.584563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:40447)
0809 02:17:23.584717 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:23.584807 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
I20260809 02:17:23.616606 26188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:23.616347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:52945)
0809 02:17:23.616511 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:23.616555 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":75,"threads_started":1}
I20260809 02:17:23.616879 26189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:23.616593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:50013)
0809 02:17:23.616805 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:23.616838 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":100,"threads_started":1}
I20260809 02:17:24.585697 26190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:24.585453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:37181)
0809 02:17:24.585607 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:24.585645 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260809 02:17:24.617414 26191 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:24.617138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:55661)
0809 02:17:24.617331 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:24.617364 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":141,"threads_started":1}
I20260809 02:17:24.617686 26192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:24.617412 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:56987)
0809 02:17:24.617608 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:24.617644 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":103,"threads_started":1}
I20260809 02:17:25.586441 26193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:25.586168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:34687)
0809 02:17:25.586328 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:25.586379 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":66,"threads_started":1}
I20260809 02:17:25.618261 26194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:25.617962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:57885)
0809 02:17:25.618139 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:25.618192 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":84,"threads_started":1}
I20260809 02:17:25.618357 26196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:25.618145 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:48213)
0809 02:17:25.618277 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:25.618314 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260809 02:17:26.587152 26197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:26.586932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:41301)
0809 02:17:26.587067 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:26.587104 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":69,"threads_started":1}
I20260809 02:17:26.619007 26199 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:26.618790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:35317)
0809 02:17:26.618929 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:26.618964 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":82,"threads_started":1}
I20260809 02:17:26.619031 26198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:26.618766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:34599)
0809 02:17:26.618903 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:26.618949 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":62,"threads_started":1}
I20260809 02:17:27.587934 26201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:27.587668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:45069)
0809 02:17:27.587836 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:27.587881 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":148,"thread_start_us":69,"threads_started":1}
I20260809 02:17:27.619844 26202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:27.619580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:36591)
0809 02:17:27.619764 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:27.619797 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":98,"threads_started":1}
I20260809 02:17:27.619848 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:27.619596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:42673)
0809 02:17:27.619760 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:27.619802 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":80,"threads_started":1}
I20260809 02:17:28.588682 26204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:28.588451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.66:51833)
0809 02:17:28.588598 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:28.588633 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":84,"threads_started":1}
I20260809 02:17:28.620558 26205 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:28.620281 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.67:40869)
0809 02:17:28.620448 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:28.620489 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
I20260809 02:17:28.620669 26206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:28.620448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43661 (local address 127.9.40.65:38769)
0809 02:17:28.620561 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:28.620600 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:43661: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":69,"threads_started":1}
I20260809 02:17:28.851255 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25668
I20260809 02:17:28.857741 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25842
I20260809 02:17:28.864411 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 25990
2026-08-09T02:17:28Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18412 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:17:28 dist-test-slave-qg60 krb5kdc[26209](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:17:28 dist-test-slave-qg60 krb5kdc[26209](info): set up 2 sockets
Aug 09 02:17:28 dist-test-slave-qg60 krb5kdc[26209](info): commencing operation
krb5kdc: starting...
W20260809 02:17:30.013686 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:17:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:17:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:17:30.149387 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:30.233249 26222 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:30.233537 26222 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:30.233582 26222 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:30.234824 26222 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:17:30.234905 26222 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:17:30.234938 26222 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:17:30.234963 26222 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:30.235018 26222 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:30.235050 26222 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:30.235080 26222 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:30.235108 26222 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:30.236783 26222 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26222
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:30.237108 26222 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:30.237377 26222 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:30.243016 26227 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.243104 26228 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.243027 26230 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:30.243114 26222 server_base.cc:1034] running on GCE node
I20260809 02:17:30.243610 26222 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:30.243947 26222 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:30.245095 26222 hybrid_clock.cc:648] HybridClock initialized: now 1786241850245082 us; error 30 us; skew 500 ppm
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:30.247741 26222 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:30.248229 26222 webserver.cc:460] Webserver started at http://127.9.40.126:42239/ using document root <none> and password file <none>
I20260809 02:17:30.248436 26222 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:30.248478 26222 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:30.248570 26222 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:30.249677 26222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "26f92bbd9ea84af6bc41ee8a89f4b948"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "2d9999885fc7b3b3ed0077a7601b8c26"
server_key_iv: "690a4d9bebf4973a3b4b22b7ec86568e"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.249989 26222 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "26f92bbd9ea84af6bc41ee8a89f4b948"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "2d9999885fc7b3b3ed0077a7601b8c26"
server_key_iv: "690a4d9bebf4973a3b4b22b7ec86568e"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.251302 26222 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:17:30.252017 26238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.252239 26222 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:30.252351 26222 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "26f92bbd9ea84af6bc41ee8a89f4b948"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "2d9999885fc7b3b3ed0077a7601b8c26"
server_key_iv: "690a4d9bebf4973a3b4b22b7ec86568e"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.252431 26222 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:30.272179 26222 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:30.274385 26222 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:30.274560 26222 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:30.279227 26222 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41175
I20260809 02:17:30.279268 26291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41175 every 8 connection(s)
I20260809 02:17:30.279642 26222 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:30.280596 26292 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:30.283272 26292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap starting.
I20260809 02:17:30.283941 26292 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:30.284253 26292 log.cc:824] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:30.285122 26292 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: No bootstrap required, opened a new log
I20260809 02:17:30.285349 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26222
I20260809 02:17:30.285457 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:17:30.285705 9377 external_mini_cluster.cc:1428] Setting key 07b3b3a275ed9999c72a5d8d4a31a60c
I20260809 02:17:30.287318 26292 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:30.287528 26292 raft_consensus.cc:374] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:30.287576 26292 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Initialized, Role: FOLLOWER
I20260809 02:17:30.287719 26292 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:30.287807 26292 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:30.287851 26292 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:30.287904 26292 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:30.288669 26292 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:30.288815 26292 leader_election.cc:302] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26f92bbd9ea84af6bc41ee8a89f4b948; no voters:
I20260809 02:17:30.289028 26292 leader_election.cc:288] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:17:30.289080 26297 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:30.289283 26297 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 LEADER]: Becoming Leader. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Running, Role: LEADER
I20260809 02:17:30.289304 26292 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:30.289383 26297 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:30.289841 26297 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26f92bbd9ea84af6bc41ee8a89f4b948. Latest consensus state: current_term: 1 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:17:30.289950 26298 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:17:30.289996 26297 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:30.290052 26298 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:30.290300 26306 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:30.291123 26306 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:30.293309 26306 catalog_manager.cc:1347] Generated new cluster ID: fa5bd6b568754c14bd0e85e6b4f35758
I20260809 02:17:30.293375 26306 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:30.299010 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.286236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57394 (local address 127.9.40.126:41175)
0809 02:17:30.286407 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.286416 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.286442 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:17:30.287300 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.287307 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.287375 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.287486 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:30.288325 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.290125 (+ 1800us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.291374 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.291991 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.292096 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.295241 (+ 3145us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:30.295245 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:30.295246 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:30.295257 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:30.297280 (+ 2023us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.297770 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.297773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.297773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.297896 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.298117 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.298118 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.298118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.298408 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:30.298555 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.298687 (+ 132us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.298817 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":72,"threads_started":1}
I20260809 02:17:30.303561 26306 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:17:30.304088 26306 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:30.313033 26306 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Generated new TSK 0
I20260809 02:17:30.313297 26306 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:17:30.345089 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:30.430661 26320 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:30.430881 26320 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:30.430907 26320 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:30.432129 26320 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:30.432190 26320 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:30.432217 26320 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:30.433744 26320 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26320
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:30.434016 26320 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:30.434281 26320 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:30.439850 26325 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.439929 26328 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.439857 26326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:30.440346 26320 server_base.cc:1034] running on GCE node
I20260809 02:17:30.440651 26320 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:30.440969 26320 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:30.442152 26320 hybrid_clock.cc:648] HybridClock initialized: now 1786241850442125 us; error 40 us; skew 500 ppm
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:30.445027 26320 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:30.445564 26320 webserver.cc:460] Webserver started at http://127.9.40.65:39941/ using document root <none> and password file <none>
I20260809 02:17:30.445822 26320 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:30.445897 26320 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:30.446067 26320 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:30.447237 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "e5892b6179004dc784653ba46b531f26"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "5776ffcb27d3a8575fe5d007fffe6b35"
server_key_iv: "926dd9b1b7a1fe2bebd90e736ab805aa"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.447636 26320 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "e5892b6179004dc784653ba46b531f26"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "5776ffcb27d3a8575fe5d007fffe6b35"
server_key_iv: "926dd9b1b7a1fe2bebd90e736ab805aa"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.449298 26320 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:17:30.450169 26336 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.450390 26320 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:30.450516 26320 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "e5892b6179004dc784653ba46b531f26"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "5776ffcb27d3a8575fe5d007fffe6b35"
server_key_iv: "926dd9b1b7a1fe2bebd90e736ab805aa"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.450603 26320 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:30.486689 26320 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:30.488945 26320 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:30.489110 26320 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:30.489400 26320 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:30.489711 26320 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:30.489746 26320 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.489771 26320 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:30.489811 26320 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.496553 26320 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37125
I20260809 02:17:30.496624 26455 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37125 every 8 connection(s)
I20260809 02:17:30.497010 26320 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:30.501415 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26320
I20260809 02:17:30.501554 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:17:30.501785 9377 external_mini_cluster.cc:1428] Setting key 7d5cd5e10df9827d75cffa2dd5d4411f
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:30.508147 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.497881 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:39245 (local address 127.9.40.126:41175)
0809 02:17:30.497959 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.497961 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.498157 (+ 196us) server_negotiation.cc:407] Connection header received
0809 02:17:30.498706 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.498708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.498722 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.498793 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:30.499832 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.500341 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.501671 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.502015 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.502121 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.504895 (+ 2774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:30.504897 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:30.504898 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:30.504914 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:30.506608 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.507026 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.507028 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.507029 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.507109 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.507373 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.507374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.507374 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.507534 (+ 160us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:30.507617 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.507886 (+ 269us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.508032 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:17:30.508170 26458 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.497797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:39245)
0809 02:17:30.497997 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.498055 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.498578 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.498857 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.498860 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.499517 (+ 657us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:30.499731 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.499736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.500437 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.500437 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.501524 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.501528 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.502154 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.502155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.502335 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.502988 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:30.502997 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:30.504780 (+ 1783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:30.506701 (+ 1921us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.506708 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.506710 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.506945 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.507190 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.507190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.507191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.507300 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.507610 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:30.507615 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:30.507741 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.507836 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.507971 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":95,"threads_started":1}
I20260809 02:17:30.508841 26456 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:30.508958 26456 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:30.509181 26456 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:30.509768 26255 ts_manager.cc:194] Registered new tserver with Master: e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:30.510433 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:39245
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:17:30.533389 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:30.621589 26461 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:30.621814 26461 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:30.621840 26461 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:30.623085 26461 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:30.623137 26461 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:30.623167 26461 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:30.624697 26461 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26461
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:30.625020 26461 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:30.625269 26461 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:30.630730 26467 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.630766 26470 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.630746 26468 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:30.631139 26461 server_base.cc:1034] running on GCE node
I20260809 02:17:30.631367 26461 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:30.631646 26461 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:30.632789 26461 hybrid_clock.cc:648] HybridClock initialized: now 1786241850632747 us; error 66 us; skew 500 ppm
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:30.635471 26461 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:30.636049 26461 webserver.cc:460] Webserver started at http://127.9.40.66:42141/ using document root <none> and password file <none>
I20260809 02:17:30.636308 26461 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:30.636402 26461 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:30.636585 26461 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:30.637748 26461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "ddc6b253cea4411f9012bf2d38c226ea"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "56f19b483829531a8fac23d5cd32b0c9"
server_key_iv: "34b09ac0d3deafa151d2414a86b8326b"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.638128 26461 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "ddc6b253cea4411f9012bf2d38c226ea"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "56f19b483829531a8fac23d5cd32b0c9"
server_key_iv: "34b09ac0d3deafa151d2414a86b8326b"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.639575 26461 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:17:30.640377 26479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.640591 26461 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:30.640699 26461 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "ddc6b253cea4411f9012bf2d38c226ea"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "56f19b483829531a8fac23d5cd32b0c9"
server_key_iv: "34b09ac0d3deafa151d2414a86b8326b"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.640822 26461 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:30.652392 26461 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:30.655112 26461 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:30.655282 26461 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:30.655534 26461 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:30.655867 26461 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:30.655921 26461 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.655963 26461 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:30.656011 26461 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.662472 26461 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45725
I20260809 02:17:30.662518 26601 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45725 every 8 connection(s)
I20260809 02:17:30.662875 26461 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:30.669152 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26461
I20260809 02:17:30.669281 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:17:30.669485 9377 external_mini_cluster.cc:1428] Setting key 7cdbb16212037930a58609ffe7189ae3
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:30.673497 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.663708 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:34273 (local address 127.9.40.126:41175)
0809 02:17:30.663786 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.663788 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.663997 (+ 209us) server_negotiation.cc:407] Connection header received
0809 02:17:30.664488 (+ 491us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.664489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.664499 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.664568 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:30.665595 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.666104 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.667181 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.667374 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.667434 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.670243 (+ 2809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:30.670244 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:30.670244 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:30.670251 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:30.671678 (+ 1427us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.672272 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.672273 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.672274 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.672347 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.672634 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.672635 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.672635 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.672811 (+ 176us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:30.672887 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.673258 (+ 371us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.673355 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:17:30.673573 26604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.663676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:34273)
0809 02:17:30.663858 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.663902 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.664396 (+ 494us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.664622 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.664625 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.665262 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:30.665476 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.665482 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.666220 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.666222 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.667087 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.667090 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.667487 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.667488 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.667618 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.668286 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:30.668296 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:30.670120 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:30.671769 (+ 1649us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.671778 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.671781 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.672176 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.672428 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.672429 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.672430 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.672563 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.672889 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:30.672895 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:30.673114 (+ 219us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.673216 (+ 102us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.673356 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":110,"threads_started":1}
I20260809 02:17:30.674108 26602 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:30.674217 26602 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:30.674414 26602 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:30.674844 26255 ts_manager.cc:194] Registered new tserver with Master: ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725)
I20260809 02:17:30.675239 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:34273
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:17:30.701156 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:30.787842 26607 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:30.788095 26607 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:30.788136 26607 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:30.789510 26607 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:30.789577 26607 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:30.789621 26607 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:30.791446 26607 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26607
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:30.791738 26607 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:30.791989 26607 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:30.797389 26612 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:30.797399 26613 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:30.797407 26607 server_base.cc:1034] running on GCE node
W20260809 02:17:30.797390 26615 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:30.797909 26607 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:30.798182 26607 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:30.799363 26607 hybrid_clock.cc:648] HybridClock initialized: now 1786241850799351 us; error 28 us; skew 500 ppm
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:30.802273 26607 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:30.802780 26607 webserver.cc:460] Webserver started at http://127.9.40.67:41385/ using document root <none> and password file <none>
I20260809 02:17:30.803035 26607 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:30.803107 26607 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:30.803257 26607 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:17:30.804354 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "365ee6706d7c493094e5cdce6b9b731f"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "65c02329c15631161e54772c66e35efd"
server_key_iv: "b5f25d40c8a41f9cd6f58514bc8b093a"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.804798 26607 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "365ee6706d7c493094e5cdce6b9b731f"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "65c02329c15631161e54772c66e35efd"
server_key_iv: "b5f25d40c8a41f9cd6f58514bc8b093a"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.806222 26607 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:30.807179 26622 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.807379 26607 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:30.807511 26607 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "365ee6706d7c493094e5cdce6b9b731f"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "65c02329c15631161e54772c66e35efd"
server_key_iv: "b5f25d40c8a41f9cd6f58514bc8b093a"
server_key_version: "encryptionkey@0"
I20260809 02:17:30.807626 26607 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:30.817170 26607 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:30.819121 26607 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:30.819239 26607 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:30.819429 26607 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:30.819761 26607 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:30.819819 26607 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.819864 26607 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:30.819890 26607 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:30.825928 26607 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:33219
I20260809 02:17:30.826007 26747 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:33219 every 8 connection(s)
I20260809 02:17:30.826357 26607 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:30.827225 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26607
I20260809 02:17:30.827327 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:17:30.827530 9377 external_mini_cluster.cc:1428] Setting key 4fea0903eb7c1b3c347e5d064cc974d7
Aug 09 02:17:30 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241850, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:30.836061 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.827337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41921 (local address 127.9.40.126:41175)
0809 02:17:30.827399 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.827402 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.827662 (+ 260us) server_negotiation.cc:407] Connection header received
0809 02:17:30.828245 (+ 583us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.828247 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.828258 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.828318 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:30.829364 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.829879 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.830890 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.831095 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.831173 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.833433 (+ 2260us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:30.833435 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:30.833435 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:30.833442 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:30.834663 (+ 1221us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.835110 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.835111 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.835112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.835169 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.835408 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.835408 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.835409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.835518 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:30.835572 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.835834 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.835916 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260809 02:17:30.836123 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.827302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:41921)
0809 02:17:30.827484 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.827529 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.828140 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.828397 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.828401 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.829065 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:30.829264 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.829268 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.829991 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.829992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.830779 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.830782 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.831194 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.831195 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.831326 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.831816 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:30.831824 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:30.833311 (+ 1487us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:30.834739 (+ 1428us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.834746 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.834747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.834970 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.835242 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:30.835242 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:30.835243 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:30.835349 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:30.835578 (+ 229us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:30.835582 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:30.835700 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.835790 (+ 90us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.835915 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260809 02:17:30.836597 26748 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:30.836688 26748 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:30.836897 26748 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:30.837268 26255 ts_manager.cc:194] Registered new tserver with Master: 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219)
I20260809 02:17:30.837666 26255 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:41921
I20260809 02:17:30.839335 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:30.847100 26296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.840810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57404 (local address 127.9.40.126:41175)
0809 02:17:30.840863 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.840865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.840880 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:17:30.840935 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.840935 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.840943 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.840991 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:30.841558 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.842041 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.842885 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.843098 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.843157 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.844517 (+ 1360us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:30.844519 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:30.844519 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:30.844525 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:30.846004 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.846374 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.846374 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.846375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.846426 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:30.846573 (+ 147us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:30.846573 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:30.846574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:30.846715 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:30.846790 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.846904 (+ 114us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.846963 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:17:30.848629 26255 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57404:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:30.850376 26255 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:30.857692 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.853041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.9.40.65:37125)
0809 02:17:30.853259 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.853265 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.853562 (+ 297us) server_negotiation.cc:407] Connection header received
0809 02:17:30.853611 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.853613 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.853630 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.853891 (+ 261us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:30.854130 (+ 239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855044 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.856647 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857210 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857317 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857379 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.857554 (+ 175us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.857595 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":91,"threads_started":1}
I20260809 02:17:30.857717 26764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.852912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.0.0.1:40142)
0809 02:17:30.853425 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.853433 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.853501 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.853982 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.853983 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.853985 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:30.854025 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.854026 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855169 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.855170 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.856530 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.856534 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857322 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857324 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.857411 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857425 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.857526 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.857590 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":497,"thread_start_us":100,"threads_started":1}
I20260809 02:17:30.857735 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.853185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.0.0.1:32818)
0809 02:17:30.853616 (+ 431us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.853628 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.853679 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.853909 (+ 230us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.853910 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.853912 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:30.853952 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.853954 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855277 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.855278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.856389 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.856391 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857420 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.857505 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857510 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.857563 (+ 53us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.857611 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"mutex_wait_us":30,"thread_start_us":89,"threads_started":1}
I20260809 02:17:30.857885 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.853116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49310 (local address 127.9.40.67:33219)
0809 02:17:30.853306 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.853310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.853319 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:30.853386 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.853388 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.853404 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.853501 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:30.853763 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855019 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.856629 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857333 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857394 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857442 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.857731 (+ 289us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.857780 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260809 02:17:30.857992 26761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.852904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33219 (local address 127.0.0.1:49310)
0809 02:17:30.853068 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:30.853112 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:30.853215 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:30.853515 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:30.853518 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:30.853520 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:30.853693 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.853696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855189 (+ 1493us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.855190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.856551 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:30.856553 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857471 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857471 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:30.857540 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857544 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:30.857590 (+ 46us) client_negotiation.cc:240] Negotiation successful
0809 02:17:30.857621 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"mutex_wait_us":72,"thread_start_us":82,"threads_started":1}
I20260809 02:17:30.858139 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:30.853453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32818 (local address 127.9.40.66:45725)
0809 02:17:30.853671 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:30.853675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:30.853686 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:30.853744 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:30.853746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:30.853761 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:30.853833 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:30.854047 (+ 214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.855165 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.856475 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:30.857236 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:30.857319 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:30.857380 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:30.857976 (+ 596us) server_negotiation.cc:299] Negotiation successful
0809 02:17:30.858037 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":100,"threads_started":1}
I20260809 02:17:30.858810 26675 tablet_service.cc:1467] Processing CreateTablet for tablet 12d92540f3694b7291186d27e3833777 (DEFAULT_TABLE table=test-table [id=1e42c5fb64e44c7d8c1c756641946190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:30.858832 26530 tablet_service.cc:1467] Processing CreateTablet for tablet 12d92540f3694b7291186d27e3833777 (DEFAULT_TABLE table=test-table [id=1e42c5fb64e44c7d8c1c756641946190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:30.859002 26388 tablet_service.cc:1467] Processing CreateTablet for tablet 12d92540f3694b7291186d27e3833777 (DEFAULT_TABLE table=test-table [id=1e42c5fb64e44c7d8c1c756641946190]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:30.859208 26530 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d92540f3694b7291186d27e3833777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:30.859210 26675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d92540f3694b7291186d27e3833777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:30.859318 26388 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 12d92540f3694b7291186d27e3833777. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:30.862120 26767 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Bootstrap starting.
I20260809 02:17:30.862653 26770 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Bootstrap starting.
I20260809 02:17:30.862946 26767 tablet_bootstrap.cc:654] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:30.863445 26770 tablet_bootstrap.cc:654] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:30.863575 26767 log.cc:824] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:30.864006 26770 log.cc:824] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:30.864379 26768 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Bootstrap starting.
I20260809 02:17:30.864418 26767 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: No bootstrap required, opened a new log
I20260809 02:17:30.864478 26767 ts_tablet_manager.cc:1397] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:30.864717 26770 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: No bootstrap required, opened a new log
I20260809 02:17:30.864831 26770 ts_tablet_manager.cc:1397] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:30.865111 26768 tablet_bootstrap.cc:654] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:30.865694 26768 log.cc:824] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:30.866585 26768 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: No bootstrap required, opened a new log
I20260809 02:17:30.866657 26768 ts_tablet_manager.cc:1397] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260809 02:17:30.866765 26767 raft_consensus.cc:348] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.866957 26767 raft_consensus.cc:374] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:30.866853 26770 raft_consensus.cc:348] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.866995 26767 raft_consensus.cc:729] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365ee6706d7c493094e5cdce6b9b731f, State: Initialized, Role: FOLLOWER
I20260809 02:17:30.867031 26770 raft_consensus.cc:374] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:30.867113 26770 raft_consensus.cc:729] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc6b253cea4411f9012bf2d38c226ea, State: Initialized, Role: FOLLOWER
I20260809 02:17:30.867159 26767 consensus_queue.cc:260] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.867237 26770 consensus_queue.cc:260] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.867491 26767 ts_tablet_manager.cc:1428] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:17:30.867523 26748 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
I20260809 02:17:30.867563 26770 ts_tablet_manager.cc:1428] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260809 02:17:30.867677 26602 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
I20260809 02:17:30.868829 26768 raft_consensus.cc:348] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.869043 26768 raft_consensus.cc:374] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:30.869096 26768 raft_consensus.cc:729] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5892b6179004dc784653ba46b531f26, State: Initialized, Role: FOLLOWER
I20260809 02:17:30.869205 26768 consensus_queue.cc:260] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:30.869407 26768 ts_tablet_manager.cc:1428] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:30.869485 26456 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
W20260809 02:17:30.913491 26603 tablet.cc:2369] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:30.997848 26457 tablet.cc:2369] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:17:31.077054 26749 tablet.cc:2369] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:31.127372 26774 raft_consensus.cc:482] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:31.127516 26774 raft_consensus.cc:504] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:31.128010 26774 leader_election.cc:288] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:31.130921 26750 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.128244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.9.40.67:37311)
0809 02:17:31.128311 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:31.128323 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:31.128397 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:31.128577 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:31.128578 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:31.128580 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:31.128661 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:31.128662 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.129360 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.129363 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:31.130106 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:31.130108 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.130642 (+ 534us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:31.130732 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:31.130735 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:17:31.130805 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:17:31.130848 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46}
I20260809 02:17:31.130920 26762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.128356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:37311 (local address 127.9.40.65:37125)
0809 02:17:31.128432 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:31.128434 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:31.128446 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:17:31.128479 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:31.128487 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:31.128498 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:31.128570 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:31.128760 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.129261 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130203 (+ 942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.130558 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130643 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:31.130654 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:31.130812 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:17:31.130850 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:17:31.131031 26765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.128419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48455 (local address 127.9.40.66:45725)
0809 02:17:31.128473 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:31.128475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:31.128526 (+ 51us) server_negotiation.cc:407] Connection header received
0809 02:17:31.128609 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:31.128609 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:31.128620 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:31.128690 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:31.128874 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.129346 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130277 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.130676 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130765 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:31.130775 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:31.130928 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:17:31.130965 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:17:31.131052 26777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.128272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.9.40.67:48455)
0809 02:17:31.128462 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:31.128473 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:31.128540 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:31.128695 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:31.128695 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:31.128697 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:31.128796 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:31.128797 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.129435 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.129436 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:31.130181 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:31.130182 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.130769 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.130770 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:31.130849 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:31.130853 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:31.130920 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:17:31.130962 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":105,"threads_started":1}
I20260809 02:17:31.131335 26408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d92540f3694b7291186d27e3833777" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5892b6179004dc784653ba46b531f26" is_pre_election: true
I20260809 02:17:31.131338 26552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d92540f3694b7291186d27e3833777" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea" is_pre_election: true
I20260809 02:17:31.131508 26552 raft_consensus.cc:2454] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 0.
I20260809 02:17:31.131511 26408 raft_consensus.cc:2454] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 0.
I20260809 02:17:31.131769 26623 leader_election.cc:302] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ee6706d7c493094e5cdce6b9b731f, ddc6b253cea4411f9012bf2d38c226ea; no voters:
I20260809 02:17:31.131906 26774 raft_consensus.cc:2781] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:31.131963 26774 raft_consensus.cc:482] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:31.131997 26774 raft_consensus.cc:3037] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:31.132766 26774 raft_consensus.cc:504] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:31.132921 26774 leader_election.cc:288] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 election: Requested vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:31.133136 26408 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d92540f3694b7291186d27e3833777" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5892b6179004dc784653ba46b531f26"
I20260809 02:17:31.133162 26552 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "12d92540f3694b7291186d27e3833777" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea"
I20260809 02:17:31.133221 26552 raft_consensus.cc:3037] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:31.133239 26408 raft_consensus.cc:3037] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:31.133930 26552 raft_consensus.cc:2454] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 1.
I20260809 02:17:31.134107 26408 raft_consensus.cc:2454] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 1.
I20260809 02:17:31.134172 26623 leader_election.cc:302] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ee6706d7c493094e5cdce6b9b731f, ddc6b253cea4411f9012bf2d38c226ea; no voters:
I20260809 02:17:31.134248 26774 raft_consensus.cc:2781] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:31.134400 26774 raft_consensus.cc:686] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 1 LEADER]: Becoming Leader. State: Replica: 365ee6706d7c493094e5cdce6b9b731f, State: Running, Role: LEADER
I20260809 02:17:31.134511 26774 consensus_queue.cc:237] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:31.135509 26254 catalog_manager.cc:5495] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f reported cstate change: term changed from 0 to 1, leader changed from <none> to 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67). New cstate: current_term: 1 leader_uuid: "365ee6706d7c493094e5cdce6b9b731f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } health_report { overall_health: UNKNOWN } } }
I20260809 02:17:31.145021 26763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.142361 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49316 (local address 127.9.40.67:33219)
0809 02:17:31.142436 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:31.142438 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:31.142467 (+ 29us) server_negotiation.cc:407] Connection header received
0809 02:17:31.142499 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:31.142500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:31.142516 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:31.142583 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:31.142772 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.143347 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.144136 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:31.144367 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:31.144439 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:31.144610 (+ 171us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:31.144704 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:31.144861 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:31.144876 (+ 15us) server_negotiation.cc:299] Negotiation successful
0809 02:17:31.144894 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:17:31.146781 26408 raft_consensus.cc:1261] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Refusing update from remote peer 365ee6706d7c493094e5cdce6b9b731f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:31.146828 26552 raft_consensus.cc:1261] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Refusing update from remote peer 365ee6706d7c493094e5cdce6b9b731f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:31.147169 26778 consensus_queue.cc:1035] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:31.147291 26774 consensus_queue.cc:1035] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:31.151669 26782 mvcc.cc:205] Tried to move back new op lower bound from 7316446622294585344 to 7316446622247825408. Current Snapshot: MvccSnapshot[applied={T|T < 7316446622294585344 or (T in {7316446622294585344})}]
I20260809 02:17:31.151820 26254 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:31.151907 26254 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57404:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:31.152762 26254 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Sending DeleteTablet for 3 replicas of tablet 12d92540f3694b7291186d27e3833777
I20260809 02:17:31.153136 26388 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d92540f3694b7291186d27e3833777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:40142
I20260809 02:17:31.153175 26675 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d92540f3694b7291186d27e3833777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:49310
I20260809 02:17:31.153235 26530 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d92540f3694b7291186d27e3833777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:32818
I20260809 02:17:31.153481 26790 tablet_replica.cc:330] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: stopping tablet replica
I20260809 02:17:31.153499 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26320
I20260809 02:17:31.158092 26791 tablet_replica.cc:330] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: stopping tablet replica
I20260809 02:17:31.158226 26791 raft_consensus.cc:2229] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:31.158339 26791 raft_consensus.cc:2258] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:31.158511 26792 tablet_replica.cc:330] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: stopping tablet replica
I20260809 02:17:31.158620 26792 raft_consensus.cc:2229] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:31.158735 26792 raft_consensus.cc:2258] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:31.159237 26791 ts_tablet_manager.cc:1905] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:31.159364 26792 ts_tablet_manager.cc:1905] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:31.161240 26792 ts_tablet_manager.cc:1918] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:31.161361 26792 log.cc:1196] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/12d92540f3694b7291186d27e3833777
I20260809 02:17:31.161484 26791 ts_tablet_manager.cc:1918] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:31.161628 26791 log.cc:1196] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/12d92540f3694b7291186d27e3833777
I20260809 02:17:31.161670 26792 ts_tablet_manager.cc:1939] T 12d92540f3694b7291186d27e3833777 P ddc6b253cea4411f9012bf2d38c226ea: Deleting consensus metadata
I20260809 02:17:31.161979 26791 ts_tablet_manager.cc:1939] T 12d92540f3694b7291186d27e3833777 P 365ee6706d7c493094e5cdce6b9b731f: Deleting consensus metadata
I20260809 02:17:31.162225 26239 catalog_manager.cc:4841] TS ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725): tablet 12d92540f3694b7291186d27e3833777 (table test-table [id=1e42c5fb64e44c7d8c1c756641946190]) successfully deleted
I20260809 02:17:31.162616 26239 catalog_manager.cc:4841] TS 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219): tablet 12d92540f3694b7291186d27e3833777 (table test-table [id=1e42c5fb64e44c7d8c1c756641946190]) successfully deleted
W20260809 02:17:31.163424 26240 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37125 (error 108)
I20260809 02:17:31.163628 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26461
I20260809 02:17:31.165233 26760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:31.165085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.0.0.1:40148)
0809 02:17:31.165155 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:31.165198 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:17:31.165453 26240 catalog_manager.cc:4584] TS e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 12d92540f3694b7291186d27e3833777: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37125: connect: Connection refused (error 111)
I20260809 02:17:31.177489 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26607
I20260809 02:17:31.184311 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26222
I20260809 02:17:51.191440 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--webserver_interface=127.9.40.126
--webserver_port=42239
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:17:51.276355 26794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:51.276566 26794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:51.276588 26794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:51.277784 26794 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:17:51.277829 26794 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:17:51.277843 26794 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:17:51.277855 26794 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:17:51.277896 26794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:51.277928 26794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:51.277945 26794 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:17:51.277979 26794 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:17:51.279500 26794 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=42239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:51.279767 26794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:51.280015 26794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:17:51.290063 26794 server_base.cc:1034] running on GCE node
W20260809 02:17:51.290145 26801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.290191 26800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.290130 26803 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:51.290594 26794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:51.290926 26794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:51.292088 26794 hybrid_clock.cc:648] HybridClock initialized: now 1786241871292086 us; error 45 us; skew 500 ppm
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:51.300704 26794 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:17:51.301239 26794 webserver.cc:460] Webserver started at http://127.9.40.126:42239/ using document root <none> and password file <none>
I20260809 02:17:51.301486 26794 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:51.301565 26794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:51.303048 26794 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.303761 26813 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.303952 26794 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.304059 26794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "26f92bbd9ea84af6bc41ee8a89f4b948"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "2d9999885fc7b3b3ed0077a7601b8c26"
server_key_iv: "690a4d9bebf4973a3b4b22b7ec86568e"
server_key_version: "encryptionkey@0"
I20260809 02:17:51.304476 26794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:51.326488 26794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:51.328649 26794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:51.328835 26794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:51.333200 26794 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41175
I20260809 02:17:51.333249 26869 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41175 every 8 connection(s)
I20260809 02:17:51.333678 26794 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:17:51.335326 26870 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:17:51.336004 26870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap starting.
I20260809 02:17:51.338112 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26794
I20260809 02:17:51.338414 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:37125
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=39941
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:51.347663 26870 log.cc:824] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:51.350872 26870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:51.351418 26870 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap complete.
I20260809 02:17:51.354918 26870 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:51.355233 26870 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Initialized, Role: FOLLOWER
I20260809 02:17:51.355402 26870 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:51.355482 26870 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:17:51.355535 26870 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:17:51.355579 26870 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:17:51.356947 26870 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:51.357062 26870 leader_election.cc:302] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26f92bbd9ea84af6bc41ee8a89f4b948; no voters:
I20260809 02:17:51.357231 26870 leader_election.cc:288] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260809 02:17:51.357307 26874 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:17:51.357494 26870 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:17:51.357523 26874 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 LEADER]: Becoming Leader. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Running, Role: LEADER
I20260809 02:17:51.357723 26874 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:17:51.358029 26875 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:17:51.358129 26875 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:51.358423 26876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26f92bbd9ea84af6bc41ee8a89f4b948. Latest consensus state: current_term: 2 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:17:51.358484 26876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:17:51.358633 26886 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:17:51.359433 26886 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:17:51.359568 26886 catalog_manager.cc:1259] Loaded cluster ID: fa5bd6b568754c14bd0e85e6b4f35758
I20260809 02:17:51.359604 26886 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:17:51.360411 26886 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:17:51.360600 26886 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Loaded TSK: 0
I20260809 02:17:51.375766 26886 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Generated new TSK 1
I20260809 02:17:51.375880 26886 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260809 02:17:51.443411 26872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:51.443629 26872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:51.443653 26872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:51.444876 26872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:51.444933 26872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:51.444972 26872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:17:51.446421 26872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:37125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=39941
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:51.446657 26872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:51.446894 26872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:51.451975 26904 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.452082 26902 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:51.452332 26872 server_base.cc:1034] running on GCE node
W20260809 02:17:51.451979 26901 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:51.452642 26872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:51.452944 26872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:51.454131 26872 hybrid_clock.cc:648] HybridClock initialized: now 1786241871454104 us; error 40 us; skew 500 ppm
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:51.457360 26872 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:17:51.457938 26872 webserver.cc:460] Webserver started at http://127.9.40.65:39941/ using document root <none> and password file <none>
I20260809 02:17:51.458222 26872 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:51.458283 26872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:51.459800 26872 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.460561 26914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.460774 26872 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.460871 26872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "e5892b6179004dc784653ba46b531f26"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "5776ffcb27d3a8575fe5d007fffe6b35"
server_key_iv: "926dd9b1b7a1fe2bebd90e736ab805aa"
server_key_version: "encryptionkey@0"
I20260809 02:17:51.461253 26872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:51.486510 26872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:51.489001 26872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:51.489180 26872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:51.489428 26872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:51.489962 26922 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:17:51.491420 26872 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:17:51.491472 26872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:17:51.491506 26872 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:17:51.492172 26872 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:17:51.492218 26872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:51.492262 26922 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Bootstrap starting.
I20260809 02:17:51.500463 26872 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37125
I20260809 02:17:51.500555 27042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37125 every 8 connection(s)
I20260809 02:17:51.501061 26872 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:17:51.504880 26922 log.cc:824] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:51.505945 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 26872
I20260809 02:17:51.506249 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:45725
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=42141
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:17:51.507782 26922 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:17:51.508105 26922 tablet_bootstrap.cc:492] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Bootstrap complete.
I20260809 02:17:51.509517 26922 ts_tablet_manager.cc:1397] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Time spent bootstrapping tablet: real 0.017s user 0.005s sys 0.010s
I20260809 02:17:51.511258 26922 raft_consensus.cc:348] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.511688 26922 raft_consensus.cc:729] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5892b6179004dc784653ba46b531f26, State: Initialized, Role: FOLLOWER
I20260809 02:17:51.511883 26922 consensus_queue.cc:260] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:51.512238 26922 ts_tablet_manager.cc:1428] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 02:17:51.517105 27046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.501833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:58803)
0809 02:17:51.502026 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.502077 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.502890 (+ 813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.505395 (+ 2505us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.505399 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.505830 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:51.506090 (+ 260us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.506098 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.507799 (+ 1701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.507800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.509308 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.509311 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.510203 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.510204 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.510352 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.510931 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:51.510939 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:51.512520 (+ 1581us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:51.514938 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.514945 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.514947 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.515180 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.515578 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.515579 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.515580 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.515754 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.516355 (+ 601us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:51.516362 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:51.516554 (+ 192us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.516674 (+ 120us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.516892 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":95,"threads_started":1}
I20260809 02:17:51.517104 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.501946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58803 (local address 127.9.40.126:41175)
0809 02:17:51.504913 (+ 2967us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.504924 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.504967 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:17:51.505139 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.505146 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.505211 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.505368 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.506246 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.507642 (+ 1396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.509416 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.510068 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.510220 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.512640 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.512645 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.512645 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.512659 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.514837 (+ 2178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.515272 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.515276 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.515277 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.515416 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.515821 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.515822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.515823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.516254 (+ 431us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.516367 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.516777 (+ 410us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.516912 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2900,"thread_start_us":82,"threads_started":1}
I20260809 02:17:51.517756 27044 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:51.517850 27044 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:51.518044 27044 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:51.518764 26832 ts_manager.cc:194] Registered new tserver with Master: e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:51.519939 26832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:58803
I20260809 02:17:51.520965 27044 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:17:51.527609 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.519593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58794 (local address 127.9.40.65:37125)
0809 02:17:51.519727 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.519731 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.519740 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:51.519771 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.519773 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.519785 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.519858 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.520355 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.521130 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.522221 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.522539 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.522606 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.524720 (+ 2114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.524743 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.524744 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.524753 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.526284 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.526710 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.526713 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.526714 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.526775 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.526971 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.526973 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.526973 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.527128 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.527215 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.527404 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.527498 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110,"thread_start_us":68,"threads_started":1}
I20260809 02:17:51.527626 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.519344 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.0.0.1:58794)
0809 02:17:51.519543 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.519594 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.519668 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.519878 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.519884 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.520225 (+ 341us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:51.520285 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.520288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.521199 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.521200 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.522121 (+ 921us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.522123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.522620 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.522621 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.522702 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.523171 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:51.523177 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:51.524623 (+ 1446us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:51.526375 (+ 1752us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.526383 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.526385 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.526607 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.526847 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.526848 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.526848 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.526911 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.527206 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:51.527214 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:51.527271 (+ 57us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.527366 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.527500 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":103,"threads_started":1}
I20260809 02:17:51.528038 26967 tablet_service.cc:1514] Processing DeleteTablet for tablet 12d92540f3694b7291186d27e3833777 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:31 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:58794
I20260809 02:17:51.528568 27053 tablet_replica.cc:330] stopping tablet replica
I20260809 02:17:51.528682 27053 raft_consensus.cc:2229] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:51.528790 27053 raft_consensus.cc:2258] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:51.529124 27053 ts_tablet_manager.cc:1905] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:51.531229 27053 ts_tablet_manager.cc:1918] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:51.531293 27053 log.cc:1196] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal/wals/12d92540f3694b7291186d27e3833777
I20260809 02:17:51.531589 27053 ts_tablet_manager.cc:1939] T 12d92540f3694b7291186d27e3833777 P e5892b6179004dc784653ba46b531f26: Deleting consensus metadata
I20260809 02:17:51.532155 26817 catalog_manager.cc:4841] TS e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125): tablet 12d92540f3694b7291186d27e3833777 (table test-table [id=1e42c5fb64e44c7d8c1c756641946190]) successfully deleted
W20260809 02:17:51.596695 27049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:51.596948 27049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:51.596972 27049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:51.598059 27049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:51.598098 27049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:51.598124 27049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:17:51.599493 27049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:45725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=42141
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:51.599685 27049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:51.599858 27049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:51.604830 27058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.604831 27059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.604859 27061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:51.604956 27049 server_base.cc:1034] running on GCE node
I20260809 02:17:51.605319 27049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:51.605556 27049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:51.606714 27049 hybrid_clock.cc:648] HybridClock initialized: now 1786241871606699 us; error 34 us; skew 500 ppm
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:51.609836 27049 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:17:51.610338 27049 webserver.cc:460] Webserver started at http://127.9.40.66:42141/ using document root <none> and password file <none>
I20260809 02:17:51.610602 27049 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:51.610699 27049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:51.612192 27049 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:17:51.612833 27070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.613018 27049 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:17:51.613127 27049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "ddc6b253cea4411f9012bf2d38c226ea"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "56f19b483829531a8fac23d5cd32b0c9"
server_key_iv: "34b09ac0d3deafa151d2414a86b8326b"
server_key_version: "encryptionkey@0"
I20260809 02:17:51.613500 27049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:51.650684 27049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:51.652923 27049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:51.653111 27049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:51.653375 27049 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:51.653709 27049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:51.653765 27049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.653810 27049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:51.653846 27049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.660698 27049 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45725
I20260809 02:17:51.660787 27195 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45725 every 8 connection(s)
I20260809 02:17:51.661202 27049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:17:51.664227 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27049
I20260809 02:17:51.664508 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:33219
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=41385
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:51.672279 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.662042 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:56315 (local address 127.9.40.126:41175)
0809 02:17:51.662112 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.662114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.662344 (+ 230us) server_negotiation.cc:407] Connection header received
0809 02:17:51.662934 (+ 590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.662935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.662950 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.663002 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.664069 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.664847 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.666010 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.666371 (+ 361us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.666452 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.669180 (+ 2728us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.669181 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.669182 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.669188 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.670589 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.671214 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.671214 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.671215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.671268 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.671578 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.671578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.671579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.671686 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.671736 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.672135 (+ 399us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.672210 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:17:51.672647 27198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.662011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:56315)
0809 02:17:51.662220 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.662260 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.662815 (+ 555us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.663104 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.663108 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.663733 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:51.663962 (+ 229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.663969 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.664945 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.664947 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.665912 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.665914 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.666471 (+ 557us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.666472 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.666599 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.667149 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:51.667157 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:51.669083 (+ 1926us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:51.670691 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.670700 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.670703 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.671130 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.671339 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.671340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.671340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.671507 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.671753 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:51.671760 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:51.671960 (+ 200us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.672113 (+ 153us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.672327 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":112,"threads_started":1}
I20260809 02:17:51.673236 27196 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:51.673331 27196 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:51.673519 27196 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:51.673934 26832 ts_manager.cc:194] Registered new tserver with Master: ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725)
I20260809 02:17:51.674319 26832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:56315
W20260809 02:17:51.751063 27199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:17:51.751299 27199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:17:51.751338 27199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:17:51.752496 27199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:17:51.752569 27199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:17:51.752615 27199 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:17:51.754108 27199 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:33219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=41385
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:17:51.754386 27199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:17:51.754619 27199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:17:51.759773 27205 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.759809 27206 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:17:51.759778 27208 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:17:51.760052 27199 server_base.cc:1034] running on GCE node
I20260809 02:17:51.760288 27199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:17:51.760543 27199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:17:51.761709 27199 hybrid_clock.cc:648] HybridClock initialized: now 1786241871761692 us; error 43 us; skew 500 ppm
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:17:51.764377 27199 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:17:51.764892 27199 webserver.cc:460] Webserver started at http://127.9.40.67:41385/ using document root <none> and password file <none>
I20260809 02:17:51.765118 27199 fs_manager.cc:359] Metadata directory not provided
I20260809 02:17:51.765187 27199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:17:51.766528 27199 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.767278 27215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.767448 27199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:17:51.767541 27199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "365ee6706d7c493094e5cdce6b9b731f"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "65c02329c15631161e54772c66e35efd"
server_key_iv: "b5f25d40c8a41f9cd6f58514bc8b093a"
server_key_version: "encryptionkey@0"
I20260809 02:17:51.767882 27199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:17:51.777746 27199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:17:51.779963 27199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:17:51.780133 27199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:17:51.780375 27199 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:17:51.780711 27199 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:17:51.780784 27199 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.780829 27199 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:17:51.780856 27199 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:17:51.786770 27199 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:33219
I20260809 02:17:51.786823 27338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:33219 every 8 connection(s)
I20260809 02:17:51.787159 27199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:17:51.791035 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27199
Aug 09 02:17:51 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241871, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:17:51.797356 27047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.787874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:46247 (local address 127.9.40.126:41175)
0809 02:17:51.788012 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.788015 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.788199 (+ 184us) server_negotiation.cc:407] Connection header received
0809 02:17:51.788665 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.788667 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.788678 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.788790 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.789823 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.790365 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.791661 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.792090 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.792178 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.794282 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.794283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.794284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.794292 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.795845 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.796290 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.796292 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.796292 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.796367 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.796616 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.796617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.796617 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.796794 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.796858 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.797161 (+ 303us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.797253 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:17:51.797441 27342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.787885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:46247)
0809 02:17:51.788074 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.788112 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.788573 (+ 461us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.788878 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.788882 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.789517 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:17:51.789720 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.789725 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.790448 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.790449 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.791560 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.791563 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.792178 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.792179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.792281 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.792862 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:17:51.792870 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:17:51.794201 (+ 1331us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:17:51.795925 (+ 1724us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.795931 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.795933 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.796202 (+ 269us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.796438 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:17:51.796439 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:17:51.796439 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:17:51.796547 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:17:51.796873 (+ 326us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:17:51.796879 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:17:51.797026 (+ 147us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.797118 (+ 92us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.797246 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":107,"threads_started":1}
I20260809 02:17:51.797974 27340 heartbeater.cc:347] Connected to a master server at 127.9.40.126:41175
I20260809 02:17:51.798100 27340 heartbeater.cc:464] Registering TS with master...
I20260809 02:17:51.798295 27340 heartbeater.cc:511] Master 127.9.40.126:41175 requested a full tablet report, sending...
I20260809 02:17:51.798730 26832 ts_manager.cc:194] Registered new tserver with Master: 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219)
I20260809 02:17:51.799103 26832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:46247
I20260809 02:17:51.799583 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.791498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58816 (local address 127.9.40.126:41175)
0809 02:17:51.791623 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.791625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.791744 (+ 119us) server_negotiation.cc:407] Connection header received
0809 02:17:51.791825 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.791825 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.791835 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.791888 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.792638 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.793228 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.794113 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.794417 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.794498 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.795818 (+ 1320us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.795820 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.795820 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.795828 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.797622 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.797988 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.797989 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.797989 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.798061 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.798300 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.798301 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.798301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.798432 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.798636 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.799356 (+ 720us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.799450 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":106,"thread_start_us":71,"threads_started":1}
I20260809 02:17:51.800423 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:17:51.803062 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.800822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58830 (local address 127.9.40.126:41175)
0809 02:17:51.800857 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.800859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.801002 (+ 143us) server_negotiation.cc:407] Connection header received
0809 02:17:51.801064 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.801064 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.801093 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.801140 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:51.801228 (+ 88us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.801788 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.802515 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.802700 (+ 185us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.802749 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.802873 (+ 124us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:51.802911 (+ 38us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0809 02:17:51.803020 (+ 109us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:58830: token expired
Metrics: {"server-negotiator.queue_time_us":16}
W20260809 02:17:51.803128 27344 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:58830: token expired
W20260809 02:17:51.803975 26751 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.9.40.126:41175)
I20260809 02:17:51.809803 27344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.804253 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58840 (local address 127.9.40.126:41175)
0809 02:17:51.804288 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.804290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.804303 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:17:51.804338 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.804339 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.804346 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.804394 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:17:51.804908 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.805347 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.806000 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.806174 (+ 174us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.806228 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.807401 (+ 1173us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:17:51.807402 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:17:51.807403 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:17:51.807410 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:17:51.808835 (+ 1425us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.809209 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.809210 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.809210 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.809263 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:17:51.809425 (+ 162us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:17:51.809425 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:17:51.809426 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:17:51.809517 (+ 91us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:17:51.809591 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.809684 (+ 93us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.809744 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 02:17:51.810985 26832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58840:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:17:51.812609 26832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:17:51.814945 26967 tablet_service.cc:1467] Processing CreateTablet for tablet 7699b08cdfe14411ab898bf4f5b6efc6 (DEFAULT_TABLE table=test-table [id=22da87adbc324257a342b5c3ff99428f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:51.815213 26967 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7699b08cdfe14411ab898bf4f5b6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:51.816826 26922 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Bootstrap starting.
I20260809 02:17:51.817160 26922 tablet_bootstrap.cc:654] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:51.817744 26922 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: No bootstrap required, opened a new log
I20260809 02:17:51.817804 26922 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:51.818115 26922 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.818168 26922 raft_consensus.cc:374] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:51.818233 26922 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5892b6179004dc784653ba46b531f26, State: Initialized, Role: FOLLOWER
I20260809 02:17:51.818280 26922 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.818378 26922 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260809 02:17:51.818548 27051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.814460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.0.0.1:36262)
0809 02:17:51.814670 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.814681 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.814721 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.814928 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.814929 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.814931 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:51.814978 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.814979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.816004 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.816005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.817192 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.817195 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.818227 (+ 1032us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.818228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.818311 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.818335 (+ 24us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.818401 (+ 66us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.818452 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260809 02:17:51.818590 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.814683 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35586 (local address 127.9.40.67:33219)
0809 02:17:51.814855 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.814861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.814981 (+ 120us) server_negotiation.cc:407] Connection header received
0809 02:17:51.815028 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.815030 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.815046 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.815123 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:51.815210 (+ 87us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.816234 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.817499 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.818178 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.818237 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.818282 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.818460 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.818503 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":101,"threads_started":1}
I20260809 02:17:51.818634 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.814566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33219 (local address 127.0.0.1:35586)
0809 02:17:51.814866 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.814876 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.814925 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.815118 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.815119 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.815121 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:51.815152 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.815153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.816327 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.816328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.817411 (+ 1083us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.817414 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.818319 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.818320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.818387 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.818391 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.818445 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.818487 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":64,"threads_started":1}
I20260809 02:17:51.818924 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.814567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36262 (local address 127.9.40.66:45725)
0809 02:17:51.814754 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.814759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.814769 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:17:51.814813 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.814815 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.814832 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.814924 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:51.815050 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.815921 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.817375 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.818127 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.818214 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.818279 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.818779 (+ 500us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.818823 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":116,"threads_started":1}
I20260809 02:17:51.819363 27266 tablet_service.cc:1467] Processing CreateTablet for tablet 7699b08cdfe14411ab898bf4f5b6efc6 (DEFAULT_TABLE table=test-table [id=22da87adbc324257a342b5c3ff99428f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:51.819506 27121 tablet_service.cc:1467] Processing CreateTablet for tablet 7699b08cdfe14411ab898bf4f5b6efc6 (DEFAULT_TABLE table=test-table [id=22da87adbc324257a342b5c3ff99428f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:17:51.819749 27266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7699b08cdfe14411ab898bf4f5b6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:51.819778 27121 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7699b08cdfe14411ab898bf4f5b6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:17:51.823299 27351 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Bootstrap starting.
I20260809 02:17:51.823299 27352 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Bootstrap starting.
I20260809 02:17:51.823995 27351 tablet_bootstrap.cc:654] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:51.823995 27352 tablet_bootstrap.cc:654] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Neither blocks nor log segments found. Creating new log.
I20260809 02:17:51.824698 27351 log.cc:824] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:51.824698 27352 log.cc:824] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Log is configured to *not* fsync() on all Append() calls
I20260809 02:17:51.825456 27352 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: No bootstrap required, opened a new log
I20260809 02:17:51.825531 27352 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:17:51.825685 27351 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: No bootstrap required, opened a new log
I20260809 02:17:51.825749 27351 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:17:51.827534 27352 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.827622 27351 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.827730 27352 raft_consensus.cc:374] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:51.827764 27351 raft_consensus.cc:374] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:17:51.827769 27352 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc6b253cea4411f9012bf2d38c226ea, State: Initialized, Role: FOLLOWER
I20260809 02:17:51.827811 27351 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365ee6706d7c493094e5cdce6b9b731f, State: Initialized, Role: FOLLOWER
I20260809 02:17:51.827919 27351 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.827919 27352 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.828218 27352 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:17:51.828218 27351 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:17:51.828284 27340 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
I20260809 02:17:51.828402 27196 heartbeater.cc:503] Master 127.9.40.126:41175 was elected leader, sending a full tablet report...
W20260809 02:17:51.911830 27197 tablet.cc:2369] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:17:51.930558 27355 raft_consensus.cc:482] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:17:51.930713 27355 raft_consensus.cc:504] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.931238 27355 leader_election.cc:288] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:51.934010 27342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.931389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.9.40.67:53275)
0809 02:17:51.931453 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.931466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.931506 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.931670 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.931672 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.931674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:51.931744 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.931746 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.932394 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.932396 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.933174 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.933176 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.933740 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.933741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.933829 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.933832 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.933896 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.933938 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:17:51.934010 27347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.931495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53275 (local address 127.9.40.66:45725)
0809 02:17:51.931546 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.931552 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.931561 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:17:51.931591 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.931592 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.931604 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.931664 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:51.931827 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.932310 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.933268 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.933656 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.933741 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.933751 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.933902 (+ 151us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.933941 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:17:51.934360 27052 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.931656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48059 (local address 127.9.40.65:37125)
0809 02:17:51.931702 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.931704 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.931715 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:17:51.931799 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.931800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.931829 (+ 29us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.931898 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:17:51.932080 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.932607 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.933565 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.933977 (+ 412us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.934066 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.934142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.934233 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.934280 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:17:51.934486 27142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea" is_pre_election: true
I20260809 02:17:51.934607 27357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.931447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.9.40.67:48059)
0809 02:17:51.931667 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.931677 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:17:51.931727 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:17:51.931926 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:17:51.931927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:17:51.931929 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:17:51.932007 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.932008 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.932696 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.932697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.933472 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:17:51.933474 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.934069 (+ 595us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.934070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:17:51.934138 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.934142 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:17:51.934471 (+ 329us) client_negotiation.cc:240] Negotiation successful
0809 02:17:51.934523 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":161,"threads_started":1}
I20260809 02:17:51.934670 27142 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 0.
I20260809 02:17:51.934887 27216 leader_election.cc:302] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ee6706d7c493094e5cdce6b9b731f, ddc6b253cea4411f9012bf2d38c226ea; no voters:
I20260809 02:17:51.934931 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5892b6179004dc784653ba46b531f26" is_pre_election: true
I20260809 02:17:51.935034 27355 raft_consensus.cc:2781] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:17:51.935088 27355 raft_consensus.cc:482] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:17:51.935082 26990 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 0.
I20260809 02:17:51.935124 27355 raft_consensus.cc:3037] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:51.935855 27355 raft_consensus.cc:504] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.936008 27355 leader_election.cc:288] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 election: Requested vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:17:51.936187 26990 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e5892b6179004dc784653ba46b531f26"
I20260809 02:17:51.936196 27142 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "365ee6706d7c493094e5cdce6b9b731f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea"
I20260809 02:17:51.936262 26990 raft_consensus.cc:3037] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:51.936285 27142 raft_consensus.cc:3037] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:17:51.937080 27142 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 1.
I20260809 02:17:51.937116 26990 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 365ee6706d7c493094e5cdce6b9b731f in term 1.
I20260809 02:17:51.937296 27217 leader_election.cc:302] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 365ee6706d7c493094e5cdce6b9b731f, e5892b6179004dc784653ba46b531f26; no voters:
I20260809 02:17:51.937409 27355 raft_consensus.cc:2781] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:17:51.937556 27355 raft_consensus.cc:686] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 1 LEADER]: Becoming Leader. State: Replica: 365ee6706d7c493094e5cdce6b9b731f, State: Running, Role: LEADER
I20260809 02:17:51.937647 27355 consensus_queue.cc:237] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:17:51.938501 26831 catalog_manager.cc:5495] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f reported cstate change: term changed from 0 to 1, leader changed from <none> to 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67). New cstate: current_term: 1 leader_uuid: "365ee6706d7c493094e5cdce6b9b731f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } health_report { overall_health: UNKNOWN } } }
I20260809 02:17:51.964577 27348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.962208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35594 (local address 127.9.40.67:33219)
0809 02:17:51.962279 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:17:51.962282 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:17:51.962289 (+ 7us) server_negotiation.cc:407] Connection header received
0809 02:17:51.962322 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:17:51.962323 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:17:51.962342 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:17:51.962416 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:17:51.962558 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.963017 (+ 459us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.963785 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:17:51.963980 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:17:51.964053 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:17:51.964217 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:17:51.964323 (+ 106us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:17:51.964416 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:17:51.964436 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 02:17:51.964456 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:17:51.966512 27142 raft_consensus.cc:1261] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Refusing update from remote peer 365ee6706d7c493094e5cdce6b9b731f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:51.966513 26990 raft_consensus.cc:1261] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Refusing update from remote peer 365ee6706d7c493094e5cdce6b9b731f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:17:51.966888 27355 consensus_queue.cc:1035] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Connected to new peer: Peer: permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:51.967048 27358 consensus_queue.cc:1035] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:17:51.969472 27360 mvcc.cc:205] Tried to move back new op lower bound from 7316446707571806208 to 7316446707457253376. Current Snapshot: MvccSnapshot[applied={T|T < 7316446707571806208}]
I20260809 02:17:51.971813 26831 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:51.971916 26831 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58840:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:17:51.972685 26831 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Sending DeleteTablet for 3 replicas of tablet 7699b08cdfe14411ab898bf4f5b6efc6
I20260809 02:17:51.973042 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26872
I20260809 02:17:51.973067 27266 tablet_service.cc:1514] Processing DeleteTablet for tablet 7699b08cdfe14411ab898bf4f5b6efc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:51 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:35586
I20260809 02:17:51.973157 27121 tablet_service.cc:1514] Processing DeleteTablet for tablet 7699b08cdfe14411ab898bf4f5b6efc6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:17:51 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:36262
I20260809 02:17:51.975538 27370 tablet_replica.cc:330] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: stopping tablet replica
I20260809 02:17:51.975651 27370 raft_consensus.cc:2229] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:17:51.975777 27370 raft_consensus.cc:2258] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:17:51.976190 27370 ts_tablet_manager.cc:1905] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:17:51.978272 27370 ts_tablet_manager.cc:1918] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:17:51.978400 27370 log.cc:1196] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/7699b08cdfe14411ab898bf4f5b6efc6
I20260809 02:17:51.978754 27370 ts_tablet_manager.cc:1939] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Deleting consensus metadata
I20260809 02:17:51.979225 26816 catalog_manager.cc:4841] TS 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219): tablet 7699b08cdfe14411ab898bf4f5b6efc6 (table test-table [id=22da87adbc324257a342b5c3ff99428f]) successfully deleted
I20260809 02:17:51.980880 27371 tablet_replica.cc:330] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: stopping tablet replica
I20260809 02:17:51.981022 27371 raft_consensus.cc:2229] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:17:51.981158 27371 raft_consensus.cc:2258] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:17:51.981370 26817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37125 (error 108)
I20260809 02:17:51.981621 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 27049
I20260809 02:17:51.982342 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.982227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.0.0.1:58810)
0809 02:17:51.982276 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.982309 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37125: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260809 02:17:51.982488 26817 catalog_manager.cc:4584] TS e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 7699b08cdfe14411ab898bf4f5b6efc6: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37125: connect: Connection refused (error 111)
I20260809 02:17:51.988359 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 27199
I20260809 02:17:51.988438 27346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:17:51.988306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.0.0.1:36276)
0809 02:17:51.988359 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:17:51.988395 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:45725: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:17:51.996953 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 26794
I20260809 02:18:12.005205 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--webserver_interface=127.9.40.126
--webserver_port=42239
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41175
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:12.088775 27373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:12.089011 27373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:12.089037 27373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:12.090322 27373 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260809 02:18:12.090372 27373 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260809 02:18:12.090385 27373 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260809 02:18:12.090397 27373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:12.090435 27373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:12.090466 27373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:12.090482 27373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:12.090508 27373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:12.092133 27373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=42239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:12.092417 27373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:12.092674 27373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:12.097781 27380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.097781 27382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.097827 27378 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:12.097863 27373 server_base.cc:1034] running on GCE node
I20260809 02:18:12.098254 27373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:12.098555 27373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:12.099706 27373 hybrid_clock.cc:648] HybridClock initialized: now 1786241892099682 us; error 40 us; skew 500 ppm
Aug 09 02:18:12 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:12.108465 27373 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:12.109082 27373 webserver.cc:460] Webserver started at http://127.9.40.126:42239/ using document root <none> and password file <none>
I20260809 02:18:12.109338 27373 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:12.109428 27373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:12.110958 27373 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:12.111831 27391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.112124 27373 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:12.112260 27373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "26f92bbd9ea84af6bc41ee8a89f4b948"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "2d9999885fc7b3b3ed0077a7601b8c26"
server_key_iv: "690a4d9bebf4973a3b4b22b7ec86568e"
server_key_version: "encryptionkey@0"
I20260809 02:18:12.112650 27373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:12.132841 27373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:12.134886 27373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:12.135000 27373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:12.139284 27373 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41175
I20260809 02:18:12.139346 27447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41175 every 8 connection(s)
I20260809 02:18:12.139742 27373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:12.141525 27448 sys_catalog.cc:263] Verifying existing consensus state
I20260809 02:18:12.141813 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27373
I20260809 02:18:12.142091 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:37125
--local_ip_for_outbound_sockets=127.9.40.65
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=39941
--webserver_interface=127.9.40.65
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:12.142330 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap starting.
I20260809 02:18:12.151855 27448 log.cc:824] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:12.154378 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:12.154722 27448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Bootstrap complete.
I20260809 02:18:12.157282 27448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:18:12.157547 27448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Initialized, Role: FOLLOWER
I20260809 02:18:12.157758 27448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:18:12.157860 27448 raft_consensus.cc:388] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:12.157923 27448 raft_consensus.cc:482] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:12.157975 27448 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 2 FOLLOWER]: Advancing to term 3
I20260809 02:18:12.159000 27448 raft_consensus.cc:504] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:18:12.159123 27448 leader_election.cc:302] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 26f92bbd9ea84af6bc41ee8a89f4b948; no voters:
I20260809 02:18:12.159341 27448 leader_election.cc:288] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260809 02:18:12.159382 27452 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 3 FOLLOWER]: Leader election won for term 3
I20260809 02:18:12.159523 27452 raft_consensus.cc:686] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [term 3 LEADER]: Becoming Leader. State: Replica: 26f92bbd9ea84af6bc41ee8a89f4b948, State: Running, Role: LEADER
I20260809 02:18:12.159633 27448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:12.159683 27452 consensus_queue.cc:237] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } }
I20260809 02:18:12.160216 27454 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 26f92bbd9ea84af6bc41ee8a89f4b948. Latest consensus state: current_term: 3 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:18:12.160317 27454 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:12.160660 27453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "26f92bbd9ea84af6bc41ee8a89f4b948" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41175 } } }
I20260809 02:18:12.160755 27453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:12.160892 27459 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:12.161787 27459 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:12.161896 27459 catalog_manager.cc:1259] Loaded cluster ID: fa5bd6b568754c14bd0e85e6b4f35758
I20260809 02:18:12.161978 27459 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:12.162933 27459 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:12.163130 27459 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Loaded TSK: 0
I20260809 02:18:12.163189 27459 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 26f92bbd9ea84af6bc41ee8a89f4b948: Loaded TSK: 1
F20260809 02:18:12.163426 27459 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7fe1b2f0b9ad google::LogMessage::Fail() at ??:0
@ 0x7fe1b2f0e88a google::LogMessage::SendToLog() at ??:0
@ 0x7fe1b2f0b4de google::LogMessage::Flush() at ??:0
@ 0x7fe1b2f0ca19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7fe1b5678104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7fe1b5675d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7fe1bae651cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7fe1bae6548e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7fe1bae65594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7fe1bae2bc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7fe1bae35ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7fe1b4dd90d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7fe1b4dce954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7fe1b5c326db start_thread at ??:0
@ 0x7fe1b1a0961f clone at ??:0
W20260809 02:18:12.251832 27450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:12.252058 27450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:12.252084 27450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:12.253371 27450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:12.253417 27450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:12.253449 27450 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:12.254963 27450 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:37125
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=39941
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:12.255209 27450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:12.255478 27450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:12.261857 27493 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.261986 27497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.261869 27494 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:12.262193 27450 server_base.cc:1034] running on GCE node
I20260809 02:18:12.262560 27450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:12.262835 27450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:12.264021 27450 hybrid_clock.cc:648] HybridClock initialized: now 1786241892263993 us; error 43 us; skew 500 ppm
Aug 09 02:18:12 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:12.266695 27450 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:18:12.267215 27450 webserver.cc:460] Webserver started at http://127.9.40.65:39941/ using document root <none> and password file <none>
I20260809 02:18:12.267484 27450 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:12.267555 27450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:12.269045 27450 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:12.269807 27504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.270020 27450 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:12.270129 27450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "e5892b6179004dc784653ba46b531f26"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "5776ffcb27d3a8575fe5d007fffe6b35"
server_key_iv: "926dd9b1b7a1fe2bebd90e736ab805aa"
server_key_version: "encryptionkey@0"
I20260809 02:18:12.270516 27450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:12.283066 27450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:12.285543 27450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:12.285713 27450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:12.285967 27450 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:12.286434 27511 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:18:12.288463 27450 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:18:12.288573 27450 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:18:12.288645 27450 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:18:12.289355 27450 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:18:12.289425 27450 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:18:12.289482 27511 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Bootstrap starting.
I20260809 02:18:12.297030 27450 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37125
I20260809 02:18:12.297211 27627 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37125 every 8 connection(s)
I20260809 02:18:12.297482 27450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:12.299274 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27450
I20260809 02:18:12.299583 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:45725
--local_ip_for_outbound_sockets=127.9.40.66
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=42141
--webserver_interface=127.9.40.66
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:12.301918 27511 log.cc:824] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:12.304224 27511 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:12.304559 27511 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Bootstrap complete.
I20260809 02:18:12.305653 27511 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Time spent bootstrapping tablet: real 0.016s user 0.001s sys 0.014s
I20260809 02:18:12.306749 27511 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:12.307008 27511 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5892b6179004dc784653ba46b531f26, State: Initialized, Role: FOLLOWER
I20260809 02:18:12.307197 27511 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:12.307497 27511 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Time spent starting tablet: real 0.002s user 0.000s sys 0.003s
W20260809 02:18:12.387574 27632 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:18:12.298262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:37615)
0809 02:18:12.298470 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.298507 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:12.298956 (+ 449us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:12.387311 (+ 88355us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":120,"threads_started":1}
W20260809 02:18:12.387892 27628 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:41175 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:41175: Client connection negotiation failed: client connection to 127.9.40.126:41175: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:18:12.388376 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.388240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:35811)
0809 02:18:12.388286 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.388341 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 02:18:12.388829 27632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.388681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:57719)
0809 02:18:12.388770 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.388807 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260809 02:18:12.388993 27628 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260809 02:18:12.405891 27633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:12.406136 27633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:12.406177 27633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:12.407342 27633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:12.407430 27633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:12.407478 27633 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:12.409040 27633 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:45725
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=42141
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:12.409300 27633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:12.409533 27633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:12.414894 27641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:12.414918 27633 server_base.cc:1034] running on GCE node
W20260809 02:18:12.414873 27640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.414873 27644 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:12.415346 27633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:12.415599 27633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:12.416793 27633 hybrid_clock.cc:648] HybridClock initialized: now 1786241892416766 us; error 59 us; skew 500 ppm
Aug 09 02:18:12 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:12.419371 27633 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:18:12.419837 27633 webserver.cc:460] Webserver started at http://127.9.40.66:42141/ using document root <none> and password file <none>
I20260809 02:18:12.420020 27633 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:12.420068 27633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:12.421445 27633 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:12.422116 27651 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.422318 27633 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:18:12.422406 27633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "ddc6b253cea4411f9012bf2d38c226ea"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "56f19b483829531a8fac23d5cd32b0c9"
server_key_iv: "34b09ac0d3deafa151d2414a86b8326b"
server_key_version: "encryptionkey@0"
I20260809 02:18:12.422753 27633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:12.440155 27633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:12.442440 27633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:12.442618 27633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:12.442909 27633 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:12.443370 27660 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260809 02:18:12.445103 27633 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260809 02:18:12.445183 27633 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260809 02:18:12.445237 27633 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260809 02:18:12.445935 27633 ts_tablet_manager.cc:610] Registered 1 tablets
I20260809 02:18:12.445996 27633 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260809 02:18:12.446089 27660 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Bootstrap starting.
I20260809 02:18:12.453420 27633 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:45725
I20260809 02:18:12.453532 27776 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:45725 every 8 connection(s)
I20260809 02:18:12.453887 27633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:12.455107 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.454593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:52077)
0809 02:18:12.454808 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.454907 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":95,"threads_started":1}
W20260809 02:18:12.455390 27777 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:41175 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:41175: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
I20260809 02:18:12.455775 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.455659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:40579)
0809 02:18:12.455711 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.455748 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:18:12.456164 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.456044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:41995)
0809 02:18:12.456099 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.456132 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:18:12.456300 27777 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:18:12.456846 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27633
I20260809 02:18:12.457248 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:33219
--local_ip_for_outbound_sockets=127.9.40.67
--tserver_master_addrs=127.9.40.126:41175
--webserver_port=41385
--webserver_interface=127.9.40.67
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38819
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:18:12.458024 27660 log.cc:824] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:12.460680 27660 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:12.460999 27660 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Bootstrap complete.
I20260809 02:18:12.462037 27660 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Time spent bootstrapping tablet: real 0.016s user 0.009s sys 0.005s
I20260809 02:18:12.463326 27660 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:12.463604 27660 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: ddc6b253cea4411f9012bf2d38c226ea, State: Initialized, Role: FOLLOWER
I20260809 02:18:12.463800 27660 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:12.464125 27660 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260809 02:18:12.464485 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.464339 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:44529)
0809 02:18:12.464400 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.464446 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:18:12.464830 27779 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.464710 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:35749)
0809 02:18:12.464774 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.464803 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260809 02:18:12.543105 27781 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:12.543341 27781 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:12.543367 27781 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:12.544533 27781 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:12.544574 27781 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:12.544600 27781 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:12.546062 27781 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38819
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:33219
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=41385
--tserver_master_addrs=127.9.40.126:41175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27781
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:12.546255 27781 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:12.546442 27781 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:12.551714 27792 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.551849 27789 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:12.551714 27790 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:12.551930 27781 server_base.cc:1034] running on GCE node
I20260809 02:18:12.552225 27781 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:12.552501 27781 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:12.553670 27781 hybrid_clock.cc:648] HybridClock initialized: now 1786241892553665 us; error 36 us; skew 500 ppm
Aug 09 02:18:12 dist-test-slave-qg60 krb5kdc[26209](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:12.556245 27781 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:18:12.556681 27781 webserver.cc:460] Webserver started at http://127.9.40.67:41385/ using document root <none> and password file <none>
I20260809 02:18:12.556941 27781 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:12.557016 27781 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:12.558421 27781 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:12.559031 27799 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.559237 27781 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:12.559345 27781 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "365ee6706d7c493094e5cdce6b9b731f"
format_stamp: "Formatted at 2026-08-09 02:17:30 on dist-test-slave-qg60"
server_key: "65c02329c15631161e54772c66e35efd"
server_key_iv: "b5f25d40c8a41f9cd6f58514bc8b093a"
server_key_version: "encryptionkey@0"
I20260809 02:18:12.559712 27781 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:12.570961 27781 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:12.572992 27781 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:12.573161 27781 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:12.573458 27781 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:12.573809 27781 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:12.573856 27781 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.573880 27781 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:12.573923 27781 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:12.580281 27781 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:33219
I20260809 02:18:12.580448 27934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:33219 every 8 connection(s)
I20260809 02:18:12.580782 27781 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:12.582002 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.581594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:59405)
0809 02:18:12.581776 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.581852 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":95,"threads_started":1}
W20260809 02:18:12.582417 27935 heartbeater.cc:644] Failed to heartbeat to 127.9.40.126:41175 (0 consecutive failures): Network error: Failed to ping master at 127.9.40.126:41175: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
I20260809 02:18:12.582746 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.582631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:43107)
0809 02:18:12.582688 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.582714 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:18:12.583025 27937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:12.582937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:59155)
0809 02:18:12.582983 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:12.583005 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:18:12.583170 27935 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260809 02:18:12.583496 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 27781
I20260809 02:18:13.389684 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.389395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:47917)
0809 02:18:13.389564 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.389614 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":75,"threads_started":1}
I20260809 02:18:13.465540 27940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.465290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:49659)
0809 02:18:13.465431 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.465475 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":69,"threads_started":1}
I20260809 02:18:13.583720 27941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.583513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:44985)
0809 02:18:13.583629 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.583662 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":61,"threads_started":1}
I20260809 02:18:13.606755 27942 raft_consensus.cc:482] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:13.606874 27942 raft_consensus.cc:504] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:13.607267 27942 leader_election.cc:288] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219)
Aug 09 02:18:13 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.66@KRBTEST.COM
Aug 09 02:18:13 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.67@KRBTEST.COM
I20260809 02:18:13.618480 27945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.607778 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:60273 (local address 127.9.40.67:33219)
0809 02:18:13.607930 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:13.607942 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:13.608011 (+ 69us) server_negotiation.cc:407] Connection header received
0809 02:18:13.608548 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:13.608550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:13.608588 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:13.608765 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:13.609180 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.610095 (+ 915us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.610871 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.611316 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.611444 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:13.614313 (+ 2869us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:13.614318 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:13.614318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:13.614333 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:13.616569 (+ 2236us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617168 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:13.617171 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:13.617171 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:13.617258 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617550 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:13.617551 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:13.617552 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:13.617802 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:13.617864 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:13.618229 (+ 365us) server_negotiation.cc:299] Negotiation successful
0809 02:18:13.618354 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":74,"threads_started":1}
I20260809 02:18:13.618467 27939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.607489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:45725 (local address 127.9.40.65:40891)
0809 02:18:13.607546 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.607558 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:13.607667 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:13.608232 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:13.608235 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:13.608546 (+ 311us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:13.608816 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:13.608822 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.609928 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.609929 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:13.610704 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:13.610706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.611312 (+ 606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.611313 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:13.611442 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:13.612103 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:13.612107 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:13.613841 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:13.616659 (+ 2818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:13.616667 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:13.616668 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:13.617018 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:13.617361 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617362 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:13.617362 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:13.617467 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:13.617919 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:13.617920 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:13.618042 (+ 122us) client_negotiation.cc:769] Sending connection context
0809 02:18:13.618154 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 02:18:13.618367 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37,"mutex_wait_us":7}
I20260809 02:18:13.618539 27944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.607672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:40891 (local address 127.9.40.66:45725)
0809 02:18:13.607825 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:13.607835 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:13.607853 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:18:13.607973 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:13.607974 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:13.608014 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:13.608187 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:13.608945 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.609820 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.610801 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.611207 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.611324 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:13.613982 (+ 2658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:13.613987 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:13.613988 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:13.614002 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:13.616561 (+ 2559us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617109 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:13.617112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:13.617113 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:13.617255 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617553 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:13.617554 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:13.617554 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:13.617814 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:13.617926 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:13.618234 (+ 308us) server_negotiation.cc:299] Negotiation successful
0809 02:18:13.618418 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":62,"threads_started":1}
I20260809 02:18:13.618907 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.607614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:33219 (local address 127.9.40.65:60273)
0809 02:18:13.607943 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.607956 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:13.608023 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:13.608721 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:13.608722 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:13.608979 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:13.609051 (+ 72us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:13.609053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.610206 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.610207 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:13.610771 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:13.610772 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:13.611408 (+ 636us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:13.611410 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:13.611469 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:13.612087 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:13.612096 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:13.614193 (+ 2097us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:13.616820 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:13.616821 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:13.616821 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:13.617078 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:13.617330 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:13.617331 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:13.617331 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:13.617465 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:13.617882 (+ 417us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:13.617889 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:13.618043 (+ 154us) client_negotiation.cc:769] Sending connection context
0809 02:18:13.618156 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:18:13.618461 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":148,"thread_start_us":78,"threads_started":1}
I20260809 02:18:13.619138 27727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "e5892b6179004dc784653ba46b531f26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea" is_pre_election: true
I20260809 02:18:13.619313 27727 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e5892b6179004dc784653ba46b531f26 in term 1.
I20260809 02:18:13.619630 27505 leader_election.cc:302] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: ddc6b253cea4411f9012bf2d38c226ea, e5892b6179004dc784653ba46b531f26; no voters:
W20260809 02:18:13.619807 27505 leader_election.cc:341] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219): Not found: Tablet not found: 7699b08cdfe14411ab898bf4f5b6efc6
I20260809 02:18:13.619937 27942 raft_consensus.cc:2781] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260809 02:18:13.620004 27942 raft_consensus.cc:482] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:13.620039 27942 raft_consensus.cc:3037] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 1 FOLLOWER]: Advancing to term 2
I20260809 02:18:13.621327 27942 raft_consensus.cc:504] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:13.621505 27942 leader_election.cc:288] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 election: Requested vote from peers ddc6b253cea4411f9012bf2d38c226ea (127.9.40.66:45725), 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219)
I20260809 02:18:13.621707 27727 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7699b08cdfe14411ab898bf4f5b6efc6" candidate_uuid: "e5892b6179004dc784653ba46b531f26" candidate_term: 2 candidate_status { last_received { term: 1 index: 2 } } ignore_live_leader: false dest_uuid: "ddc6b253cea4411f9012bf2d38c226ea"
I20260809 02:18:13.621814 27727 raft_consensus.cc:3037] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 1 FOLLOWER]: Advancing to term 2
W20260809 02:18:13.621938 27505 leader_election.cc:341] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 election: Tablet error from VoteRequest() call to peer 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219): Not found: Tablet not found: 7699b08cdfe14411ab898bf4f5b6efc6
I20260809 02:18:13.623183 27727 raft_consensus.cc:2454] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e5892b6179004dc784653ba46b531f26 in term 2.
I20260809 02:18:13.623383 27505 leader_election.cc:302] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: ddc6b253cea4411f9012bf2d38c226ea, e5892b6179004dc784653ba46b531f26; no voters: 365ee6706d7c493094e5cdce6b9b731f
I20260809 02:18:13.623492 27942 raft_consensus.cc:2781] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 2 FOLLOWER]: Leader election won for term 2
I20260809 02:18:13.623608 27942 raft_consensus.cc:686] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [term 2 LEADER]: Becoming Leader. State: Replica: e5892b6179004dc784653ba46b531f26, State: Running, Role: LEADER
I20260809 02:18:13.623703 27942 consensus_queue.cc:237] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 2, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:13.623973 27943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:13.623870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:59147)
0809 02:18:13.623908 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:13.623941 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260809 02:18:14.145519 27727 raft_consensus.cc:1261] T 7699b08cdfe14411ab898bf4f5b6efc6 P ddc6b253cea4411f9012bf2d38c226ea [term 2 FOLLOWER]: Refusing update from remote peer e5892b6179004dc784653ba46b531f26: Log matching property violated. Preceding OpId in replica: term: 1 index: 2. Preceding OpId from leader: term: 2 index: 3. (index mismatch)
I20260809 02:18:14.145998 27948 consensus_queue.cc:1035] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 3, Last known committed idx: 2, Time since last communication: 0.000s
I20260809 02:18:14.147157 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.146973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:45333)
0809 02:18:14.147077 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.147117 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":90,"thread_start_us":47,"threads_started":1}
I20260809 02:18:14.148267 27955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.147920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:41557)
0809 02:18:14.148148 (+ 228us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.148214 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":56,"threads_started":1}
I20260809 02:18:14.148296 27951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.148188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:48999)
0809 02:18:14.148228 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.148262 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260809 02:18:14.148329 27505 consensus_peers.cc:487] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26 -> Peer 365ee6706d7c493094e5cdce6b9b731f (127.9.40.67:33219): Couldn't send request to peer 365ee6706d7c493094e5cdce6b9b731f. Error code: TABLET_NOT_FOUND (6). Status: Not found: Tablet not found: 7699b08cdfe14411ab898bf4f5b6efc6. This is attempt 1: this message will repeat every 5th retry.
I20260809 02:18:14.584549 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.584330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:45557)
0809 02:18:14.584464 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.584503 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":66,"threads_started":1}
I20260809 02:18:14.635996 27957 ts_tablet_manager.cc:927] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Initiating tablet copy from peer e5892b6179004dc784653ba46b531f26 (127.9.40.65:37125)
I20260809 02:18:14.636312 27957 tablet_copy_client.cc:316] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: tablet copy: Beginning tablet copy session from remote peer at address 127.9.40.65:37125
Aug 09 02:18:14 dist-test-slave-qg60 krb5kdc[26209](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241892, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.65@KRBTEST.COM
I20260809 02:18:14.645195 27959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.636804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:43465 (local address 127.9.40.65:37125)
0809 02:18:14.636977 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:14.636981 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:14.636990 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:18:14.637014 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:14.637016 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:14.637028 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:14.637159 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:14.637731 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:14.638570 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:14.639380 (+ 810us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:14.639696 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:14.639782 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:14.642125 (+ 2343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:14.642133 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:14.642133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:14.642141 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:14.643706 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:14.644189 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:14.644192 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:14.644193 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:14.644261 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:14.644459 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:14.644460 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:14.644460 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:14.644621 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:14.644694 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:14.645030 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 02:18:14.645116 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":83,"threads_started":1}
I20260809 02:18:14.645255 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.636652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37125 (local address 127.9.40.67:43465)
0809 02:18:14.636716 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.636764 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:14.636843 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:14.637163 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:14.637171 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:14.637476 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:14.637605 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:14.637608 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:14.638666 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:14.638667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:14.639256 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:14.639258 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:14.639780 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:14.639781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:14.639873 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:14.640386 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:14.640391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:14.641982 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:14.643804 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:14.643817 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:14.643819 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:14.644093 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:14.644335 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:14.644336 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:14.644336 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:14.644397 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:14.644798 (+ 401us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:14.644814 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:14.644875 (+ 61us) client_negotiation.cc:769] Sending connection context
0809 02:18:14.644968 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:18:14.645097 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:18:14.645884 27601 tablet_copy_service.cc:140] P e5892b6179004dc784653ba46b531f26: Received BeginTabletCopySession request for tablet 7699b08cdfe14411ab898bf4f5b6efc6 from peer 365ee6706d7c493094e5cdce6b9b731f ({username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43465)
I20260809 02:18:14.645956 27601 tablet_copy_service.cc:161] P e5892b6179004dc784653ba46b531f26: Beginning new tablet copy session on tablet 7699b08cdfe14411ab898bf4f5b6efc6 from peer 365ee6706d7c493094e5cdce6b9b731f at {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43465: session id = 365ee6706d7c493094e5cdce6b9b731f-7699b08cdfe14411ab898bf4f5b6efc6
I20260809 02:18:14.646502 27601 tablet_copy_source_session.cc:205] T 7699b08cdfe14411ab898bf4f5b6efc6 P e5892b6179004dc784653ba46b531f26: Tablet Copy: opened 0 blocks and 1 log segments
I20260809 02:18:14.647589 27957 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7699b08cdfe14411ab898bf4f5b6efc6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:14.650715 27957 tablet_copy_client.cc:791] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: tablet copy: Starting download of 0 data blocks...
I20260809 02:18:14.650892 27957 tablet_copy_client.cc:655] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: tablet copy: Starting download of 1 WAL segments...
I20260809 02:18:14.651854 27957 tablet_copy_client.cc:531] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260809 02:18:14.652591 27957 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Bootstrap starting.
I20260809 02:18:14.654002 27957 log.cc:824] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:14.656424 27957 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Bootstrap replayed 1/1 log segments. Stats: ops{read=3 overwritten=0 applied=3 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260809 02:18:14.656635 27957 tablet_bootstrap.cc:492] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Bootstrap complete.
I20260809 02:18:14.656795 27957 ts_tablet_manager.cc:1397] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Time spent bootstrapping tablet: real 0.004s user 0.005s sys 0.000s
I20260809 02:18:14.657747 27957 raft_consensus.cc:348] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:14.657902 27957 raft_consensus.cc:729] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 365ee6706d7c493094e5cdce6b9b731f, State: Initialized, Role: FOLLOWER
I20260809 02:18:14.658030 27957 consensus_queue.cc:260] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 3, Last appended: 2.3, Last appended by leader: 3, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ddc6b253cea4411f9012bf2d38c226ea" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 45725 } } peers { permanent_uuid: "365ee6706d7c493094e5cdce6b9b731f" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 33219 } } peers { permanent_uuid: "e5892b6179004dc784653ba46b531f26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37125 } }
I20260809 02:18:14.658269 27957 ts_tablet_manager.cc:1428] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:14.658607 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.658496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:33267)
0809 02:18:14.658543 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.658572 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
I20260809 02:18:14.658666 27601 tablet_copy_service.cc:342] P e5892b6179004dc784653ba46b531f26: Request end of tablet copy session 365ee6706d7c493094e5cdce6b9b731f-7699b08cdfe14411ab898bf4f5b6efc6 received from {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:43465
I20260809 02:18:14.658759 27601 tablet_copy_service.cc:433] P e5892b6179004dc784653ba46b531f26: ending tablet copy session 365ee6706d7c493094e5cdce6b9b731f-7699b08cdfe14411ab898bf4f5b6efc6 on tablet 7699b08cdfe14411ab898bf4f5b6efc6 with peer 365ee6706d7c493094e5cdce6b9b731f
I20260809 02:18:14.659030 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:14.658926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:37195)
0809 02:18:14.658976 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:14.659007 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:18:15.027232 27964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:15.027072 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:40755)
0809 02:18:15.027161 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:15.027186 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77,"thread_start_us":40,"threads_started":1}
I20260809 02:18:15.027251 27881 raft_consensus.cc:1203] T 7699b08cdfe14411ab898bf4f5b6efc6 P 365ee6706d7c493094e5cdce6b9b731f [term 2 FOLLOWER]: Deduplicated request from leader. Original: 1.2->[2.3-2.3] Dedup: 2.3->[]
I20260809 02:18:15.027752 27956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:15.027615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:45481)
0809 02:18:15.027681 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:15.027720 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:18:15.149127 27965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:15.148876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:51479)
0809 02:18:15.149037 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:15.149079 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":67,"threads_started":1}
I20260809 02:18:16.028108 27966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:16.027804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:49489)
0809 02:18:16.027978 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:16.028022 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":95,"threads_started":1}
I20260809 02:18:16.028378 27967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:16.028153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:33399)
0809 02:18:16.028305 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:16.028338 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":69,"threads_started":1}
I20260809 02:18:16.149915 27968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:16.149682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:58005)
0809 02:18:16.149815 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:16.149864 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":68,"threads_started":1}
I20260809 02:18:17.029021 27969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:17.028689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:47609)
0809 02:18:17.028895 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:17.028954 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":120,"threads_started":1}
I20260809 02:18:17.029167 27970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:17.028957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:35595)
0809 02:18:17.029088 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:17.029125 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":63,"threads_started":1}
I20260809 02:18:17.150669 27971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:17.150440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:58393)
0809 02:18:17.150574 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:17.150617 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260809 02:18:18.029964 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:18.029737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:51873)
0809 02:18:18.029878 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:18.029913 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":87,"threads_started":1}
I20260809 02:18:18.029996 27973 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:18.029781 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:59325)
0809 02:18:18.029903 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:18.029955 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
I20260809 02:18:18.030395 27972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:18.030300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:55463)
0809 02:18:18.030346 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:18.030375 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:18:18.151582 27974 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:18.151289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:50405)
0809 02:18:18.151464 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:18.151519 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":64,"threads_started":1}
I20260809 02:18:19.030731 27975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:19.030479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:47679)
0809 02:18:19.030637 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:19.030683 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":70,"threads_started":1}
I20260809 02:18:19.031023 27976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:19.030808 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:58619)
0809 02:18:19.030953 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:19.030984 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
I20260809 02:18:19.152318 27977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:19.152088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:34595)
0809 02:18:19.152227 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:19.152267 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":64,"threads_started":1}
I20260809 02:18:20.031531 27978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:20.031250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:36655)
0809 02:18:20.031427 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:20.031463 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":88,"threads_started":1}
I20260809 02:18:20.031733 27979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:20.031459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:43085)
0809 02:18:20.031604 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:20.031669 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":80,"threads_started":1}
I20260809 02:18:20.153239 27980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:20.152941 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:53555)
0809 02:18:20.153102 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:20.153189 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":72,"threads_started":1}
I20260809 02:18:21.032378 27981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:21.032134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:50195)
0809 02:18:21.032292 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:21.032329 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":97,"threads_started":1}
I20260809 02:18:21.032496 27982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:21.032269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:39585)
0809 02:18:21.032404 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:21.032435 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260809 02:18:21.154023 27983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:21.153797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:40321)
0809 02:18:21.153935 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:21.153974 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":74,"threads_started":1}
I20260809 02:18:22.033079 27984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.032858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:43849)
0809 02:18:22.032998 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:22.033030 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":84,"threads_started":1}
I20260809 02:18:22.033231 27985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.032982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:38071)
0809 02:18:22.033125 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:22.033166 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":101,"threads_started":1}
I20260809 02:18:22.154764 27986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:22.154520 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:39503)
0809 02:18:22.154673 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:22.154716 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":63,"threads_started":1}
I20260809 02:18:23.033807 27987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.033553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:54847)
0809 02:18:23.033699 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:23.033743 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":116,"threads_started":1}
I20260809 02:18:23.034047 27988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.033745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:60307)
0809 02:18:23.033875 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:23.033990 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260809 02:18:23.155665 27989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:23.155400 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:60131)
0809 02:18:23.155578 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:23.155613 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":119,"threads_started":1}
I20260809 02:18:24.034614 27991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.034345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:37183)
0809 02:18:24.034477 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:24.034557 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":93,"threads_started":1}
I20260809 02:18:24.034809 27992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.034576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:46755)
0809 02:18:24.034710 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:24.034752 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":88,"threads_started":1}
I20260809 02:18:24.156394 27993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:24.156156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:51523)
0809 02:18:24.156304 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:24.156344 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":79,"threads_started":1}
I20260809 02:18:25.035386 27994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.035142 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:58657)
0809 02:18:25.035306 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.035339 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":110,"threads_started":1}
I20260809 02:18:25.035544 27995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.035269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:42783)
0809 02:18:25.035429 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.035489 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":107,"threads_started":1}
I20260809 02:18:25.157248 27996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:25.156985 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:59985)
0809 02:18:25.157152 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:25.157196 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":67,"threads_started":1}
I20260809 02:18:26.036216 27997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:26.035961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:58195)
0809 02:18:26.036129 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:26.036167 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":83,"threads_started":1}
I20260809 02:18:26.036243 27998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:26.036010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:59183)
0809 02:18:26.036158 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:26.036202 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":75,"threads_started":1}
I20260809 02:18:26.158021 27999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:26.157753 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:53235)
0809 02:18:26.157936 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:26.157973 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":128,"threads_started":1}
I20260809 02:18:27.037075 28002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:27.036812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.67:45489)
0809 02:18:27.036988 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:27.037022 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":82,"threads_started":1}
I20260809 02:18:27.037120 28000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:27.036812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.65:36053)
0809 02:18:27.037022 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:27.037071 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":85,"threads_started":1}
I20260809 02:18:27.158849 28003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:27.158561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:41175 (local address 127.9.40.66:39263)
0809 02:18:27.158745 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:27.158794 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.126:41175: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":104,"threads_started":1}
I20260809 02:18:27.586398 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 27450
I20260809 02:18:27.594738 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 27633
I20260809 02:18:27.600867 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 27781
2026-08-09T02:18:27Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58734 ms)
[----------] 30 tests from SecurityITest (237873 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:27 dist-test-slave-qg60 krb5kdc[28006](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:27 dist-test-slave-qg60 krb5kdc[28006](info): set up 2 sockets
Aug 09 02:18:27 dist-test-slave-qg60 krb5kdc[28006](info): commencing operation
krb5kdc: starting...
W20260809 02:18:28.762683 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:28 dist-test-slave-qg60 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:28.900967 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43175
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:43175
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:28.986456 28021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:28.986668 28021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:28.986692 28021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:28.987807 28021 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:28.987845 28021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:28.987860 28021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:28.987872 28021 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:28.987884 28021 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:28.989387 28021 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:43175
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:43175
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:28.989606 28021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:28.989794 28021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:28.995378 28026 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.995378 28029 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:28.995378 28027 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:28.995568 28021 server_base.cc:1034] running on GCE node
I20260809 02:18:28.996068 28021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:28.996384 28021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:28.997579 28021 hybrid_clock.cc:648] HybridClock initialized: now 1786241908997566 us; error 37 us; skew 500 ppm
Aug 09 02:18:28 dist-test-slave-qg60 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:29.000381 28021 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:29.000958 28021 webserver.cc:460] Webserver started at http://127.9.40.126:33649/ using document root <none> and password file <none>
I20260809 02:18:29.001245 28021 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:29.001318 28021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:29.001484 28021 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:29.002636 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "18c2d5372ea24d6d90058210291e1454"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-qg60"
server_key: "3f31309ca298da4ffcf04192a06874c3"
server_key_iv: "6e8fe8af04a5b7f18a5bbda74675ed4d"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.003032 28021 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "18c2d5372ea24d6d90058210291e1454"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-qg60"
server_key: "3f31309ca298da4ffcf04192a06874c3"
server_key_iv: "6e8fe8af04a5b7f18a5bbda74675ed4d"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.004578 28021 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:29.005553 28039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.005788 28021 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:29.005908 28021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "18c2d5372ea24d6d90058210291e1454"
format_stamp: "Formatted at 2026-08-09 02:18:28 on dist-test-slave-qg60"
server_key: "3f31309ca298da4ffcf04192a06874c3"
server_key_iv: "6e8fe8af04a5b7f18a5bbda74675ed4d"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.005982 28021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:29.015129 28021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:29.017642 28021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:29.017832 28021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:29.022305 28021 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:43175
I20260809 02:18:29.022343 28095 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:43175 every 8 connection(s)
I20260809 02:18:29.022723 28021 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:29.023686 28096 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:29.026141 28096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: Bootstrap starting.
I20260809 02:18:29.026835 28096 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.027074 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28021
I20260809 02:18:29.027171 28096 log.cc:824] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.027220 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:29.027598 9377 external_mini_cluster.cc:1428] Setting key 151b1ab688b2f065d6da6bb88a425ee9
I20260809 02:18:29.028288 28096 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: No bootstrap required, opened a new log
I20260809 02:18:29.030267 28096 raft_consensus.cc:348] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } }
I20260809 02:18:29.030440 28096 raft_consensus.cc:374] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.030505 28096 raft_consensus.cc:729] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 18c2d5372ea24d6d90058210291e1454, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.030673 28096 consensus_queue.cc:260] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } }
I20260809 02:18:29.030783 28096 raft_consensus.cc:388] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:29.030831 28096 raft_consensus.cc:482] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:29.030886 28096 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.031818 28096 raft_consensus.cc:504] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } }
I20260809 02:18:29.031945 28096 leader_election.cc:302] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 18c2d5372ea24d6d90058210291e1454; no voters:
I20260809 02:18:29.032169 28096 leader_election.cc:288] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:29.032199 28101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:29.032326 28101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [term 1 LEADER]: Becoming Leader. State: Replica: 18c2d5372ea24d6d90058210291e1454, State: Running, Role: LEADER
I20260809 02:18:29.032440 28101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } }
I20260809 02:18:29.032473 28096 sys_catalog.cc:564] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:29.033939 28102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "18c2d5372ea24d6d90058210291e1454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } } }
I20260809 02:18:29.034060 28102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:29.034205 28103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 18c2d5372ea24d6d90058210291e1454. Latest consensus state: current_term: 1 leader_uuid: "18c2d5372ea24d6d90058210291e1454" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "18c2d5372ea24d6d90058210291e1454" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 43175 } } }
I20260809 02:18:29.034291 28103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454 [sys.catalog]: This master's current role is: LEADER
W20260809 02:18:29.034603 28116 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:29.034823 28116 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:29.034909 28117 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:29.035169 28117 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:29.036937 28117 catalog_manager.cc:1347] Generated new cluster ID: c28ecb27855544f989ab014775d52061
I20260809 02:18:29.036998 28117 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241908, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:29.041198 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.028252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54914 (local address 127.9.40.126:43175)
0809 02:18:29.028471 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.028487 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.028513 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:18:29.030258 (+ 1745us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.030267 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.030328 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.030516 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.031281 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.032487 (+ 1206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.033713 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.034412 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.034533 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.037757 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.037761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.037761 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.037772 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.039566 (+ 1794us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.039965 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.039968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.039968 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.040122 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.040305 (+ 183us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.040305 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.040305 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.040542 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.040648 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.040820 (+ 172us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.040977 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":86,"threads_started":1}
I20260809 02:18:29.047240 28117 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:29.048051 28117 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:29.057922 28117 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: Generated new TSK 0
I20260809 02:18:29.058272 28117 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:18:29.097074 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:29.182842 28122 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:29.183058 28122 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:29.183084 28122 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:29.184280 28122 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:29.184329 28122 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:29.184355 28122 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:29.185981 28122 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28122
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:29.186264 28122 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:29.186525 28122 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:29.191828 28128 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:29.191797 28127 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:29.191797 28130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:29.192209 28122 server_base.cc:1034] running on GCE node
I20260809 02:18:29.192405 28122 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:29.192636 28122 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:29.193799 28122 hybrid_clock.cc:648] HybridClock initialized: now 1786241909193783 us; error 43 us; skew 500 ppm
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:29.196465 28122 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:18:29.196981 28122 webserver.cc:460] Webserver started at http://127.9.40.65:35575/ using document root <none> and password file <none>
I20260809 02:18:29.197243 28122 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:29.197291 28122 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:29.197449 28122 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:29.198532 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "45d1750d3a284263bd78f4697ed7a5c2"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "5c1f32132583b02655fb482700f6178f"
server_key_iv: "aa6d5572aebcaadc01ebe3ebced8c4bf"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.198906 28122 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "45d1750d3a284263bd78f4697ed7a5c2"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "5c1f32132583b02655fb482700f6178f"
server_key_iv: "aa6d5572aebcaadc01ebe3ebced8c4bf"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.200297 28122 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:29.201105 28137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.201311 28122 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:18:29.201406 28122 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "45d1750d3a284263bd78f4697ed7a5c2"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "5c1f32132583b02655fb482700f6178f"
server_key_iv: "aa6d5572aebcaadc01ebe3ebced8c4bf"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.201463 28122 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:29.221953 28122 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:29.224309 28122 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:29.224438 28122 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:29.224627 28122 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:29.225000 28122 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:29.225039 28122 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.225062 28122 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:29.225075 28122 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.231529 28122 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:42667
I20260809 02:18:29.231575 28261 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:42667 every 8 connection(s)
I20260809 02:18:29.231921 28122 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:29.233476 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28122
I20260809 02:18:29.233603 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:29.233803 9377 external_mini_cluster.cc:1428] Setting key 763518390fa99a0c7fd1620d2adc3da5
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:29.242758 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.232707 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:33655 (local address 127.9.40.126:43175)
0809 02:18:29.232774 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.232777 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.233013 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:18:29.233671 (+ 658us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.233673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.233692 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.233765 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.235101 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.235634 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.236692 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.236963 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.237058 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.239691 (+ 2633us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.239693 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.239693 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.239702 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.241263 (+ 1561us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.241696 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.241698 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.241698 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.241764 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.242023 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.242024 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.242024 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.242151 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.242235 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.242531 (+ 296us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.242659 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:18:29.242908 28264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.232690 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43175 (local address 127.9.40.65:33655)
0809 02:18:29.232880 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.232918 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.233520 (+ 602us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.233989 (+ 469us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.233992 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.234759 (+ 767us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.234992 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.234997 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.235751 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.235752 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.236592 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.236595 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.237071 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.237072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.237222 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.237805 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.237813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.239576 (+ 1763us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.241367 (+ 1791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.241376 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.241378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.241617 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.241838 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.241839 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.241839 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.241953 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.242224 (+ 271us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.242232 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.242385 (+ 153us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.242491 (+ 106us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.242664 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
I20260809 02:18:29.243510 28262 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43175
I20260809 02:18:29.243630 28262 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:29.243821 28262 heartbeater.cc:511] Master 127.9.40.126:43175 requested a full tablet report, sending...
I20260809 02:18:29.244488 28057 ts_manager.cc:194] Registered new tserver with Master: 45d1750d3a284263bd78f4697ed7a5c2 (127.9.40.65:42667)
I20260809 02:18:29.245218 28057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:33655
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:18:29.264240 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:29.348899 28267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:29.349182 28267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:29.349224 28267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:29.350426 28267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:29.350499 28267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:29.350543 28267 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:29.352067 28267 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:29.352360 28267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:29.352619 28267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:29.357810 28276 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:29.357805 28273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:29.357805 28272 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:29.358037 28267 server_base.cc:1034] running on GCE node
I20260809 02:18:29.358318 28267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:29.358589 28267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:29.359754 28267 hybrid_clock.cc:648] HybridClock initialized: now 1786241909359742 us; error 32 us; skew 500 ppm
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:29.362584 28267 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:18:29.363065 28267 webserver.cc:460] Webserver started at http://127.9.40.66:35521/ using document root <none> and password file <none>
I20260809 02:18:29.363278 28267 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:29.363319 28267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:29.363415 28267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:29.364468 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "1fa2f38075564115a65e5e4d0d2c7bf1"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "da3df4877b781284d4b9599c17b61b1a"
server_key_iv: "d5748e155d5243dd882474a7a96cf564"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.364845 28267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "1fa2f38075564115a65e5e4d0d2c7bf1"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "da3df4877b781284d4b9599c17b61b1a"
server_key_iv: "d5748e155d5243dd882474a7a96cf564"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.366205 28267 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:29.366963 28283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.367194 28267 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:29.367311 28267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "1fa2f38075564115a65e5e4d0d2c7bf1"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "da3df4877b781284d4b9599c17b61b1a"
server_key_iv: "d5748e155d5243dd882474a7a96cf564"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.367395 28267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:29.379170 28267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:29.381762 28267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:29.381953 28267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:29.382211 28267 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:29.382546 28267 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:29.382604 28267 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.382647 28267 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:29.382694 28267 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.389153 28267 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:43203
I20260809 02:18:29.389215 28406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:43203 every 8 connection(s)
I20260809 02:18:29.389573 28267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:29.390568 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28267
I20260809 02:18:29.390683 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:29.390913 9377 external_mini_cluster.cc:1428] Setting key f017dead515238aefe9373b63d9c3130
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:29.400949 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.390442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35467 (local address 127.9.40.126:43175)
0809 02:18:29.390525 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.390528 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.390761 (+ 233us) server_negotiation.cc:407] Connection header received
0809 02:18:29.391476 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.391477 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.391488 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.391541 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.392879 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.393383 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.394530 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.394842 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.394924 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.397724 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.397725 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.397725 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.397733 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.399207 (+ 1474us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.399790 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.399791 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.399791 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.399841 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.400166 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.400167 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.400167 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.400275 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.400324 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.400808 (+ 484us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.400880 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:18:29.401206 28411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.390362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43175 (local address 127.9.40.66:35467)
0809 02:18:29.390617 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.390676 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.391383 (+ 707us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.391600 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.391603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.392446 (+ 843us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.392778 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.392785 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.393482 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.393483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.394433 (+ 950us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.394436 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.394935 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.394936 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.395045 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.395621 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.395629 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.397616 (+ 1987us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.399302 (+ 1686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.399311 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.399313 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.399705 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.399931 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.399932 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.399932 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.400081 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.400347 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.400355 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.400540 (+ 185us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.400752 (+ 212us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.400947 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
I20260809 02:18:29.401753 28407 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43175
I20260809 02:18:29.401875 28407 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:29.402077 28407 heartbeater.cc:511] Master 127.9.40.126:43175 requested a full tablet report, sending...
I20260809 02:18:29.402513 28057 ts_manager.cc:194] Registered new tserver with Master: 1fa2f38075564115a65e5e4d0d2c7bf1 (127.9.40.66:43203)
I20260809 02:18:29.402920 28057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:35467
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:18:29.424878 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:42371
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:29.511811 28414 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:29.512036 28414 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:29.512065 28414 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:29.513257 28414 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:29.513296 28414 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:29.513324 28414 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:29.514706 28414 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42371
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:43175
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28414
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:29.514894 28414 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:29.515069 28414 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:29.520422 28421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:29.520467 28422 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:29.520671 28414 server_base.cc:1034] running on GCE node
W20260809 02:18:29.520453 28424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:29.520929 28414 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:29.521219 28414 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:29.522375 28414 hybrid_clock.cc:648] HybridClock initialized: now 1786241909522338 us; error 58 us; skew 500 ppm
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:29.525146 28414 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:18:29.525550 28414 webserver.cc:460] Webserver started at http://127.9.40.67:46143/ using document root <none> and password file <none>
I20260809 02:18:29.525734 28414 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:29.525772 28414 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:29.525863 28414 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:29.526921 28414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "38ec2587588f484aabc5a2213d89f0b4"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "897d0e3e54017f8633ea3487af99127f"
server_key_iv: "7780956e5e36750e271fd573796fb34b"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.527240 28414 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "38ec2587588f484aabc5a2213d89f0b4"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "897d0e3e54017f8633ea3487af99127f"
server_key_iv: "7780956e5e36750e271fd573796fb34b"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.528664 28414 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:29.529578 28431 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.529762 28414 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:29.529882 28414 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "38ec2587588f484aabc5a2213d89f0b4"
format_stamp: "Formatted at 2026-08-09 02:18:29 on dist-test-slave-qg60"
server_key: "897d0e3e54017f8633ea3487af99127f"
server_key_iv: "7780956e5e36750e271fd573796fb34b"
server_key_version: "encryptionkey@0"
I20260809 02:18:29.529969 28414 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:29.542614 28414 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:29.545027 28414 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:29.545214 28414 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:29.545491 28414 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:29.545843 28414 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:29.545907 28414 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.545959 28414 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:29.545996 28414 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:29.552534 28414 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:36153
I20260809 02:18:29.552570 28556 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:36153 every 8 connection(s)
I20260809 02:18:29.552999 28414 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
Aug 09 02:18:29 dist-test-slave-qg60 krb5kdc[28006](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241909, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:29.561503 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28414
I20260809 02:18:29.561640 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:29.561836 9377 external_mini_cluster.cc:1428] Setting key a35724147e2b55ac19c01ead85b33855
I20260809 02:18:29.563933 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.553791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:35977 (local address 127.9.40.126:43175)
0809 02:18:29.553880 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.553882 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.554118 (+ 236us) server_negotiation.cc:407] Connection header received
0809 02:18:29.554662 (+ 544us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.554663 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.554674 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.554722 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.555752 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.556261 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.557463 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.557740 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.557800 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.560440 (+ 2640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.560441 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.560441 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.560449 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.562209 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.562801 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.562802 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.562803 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.562872 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.563198 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.563199 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.563199 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.563366 (+ 167us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.563435 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.563754 (+ 319us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.563840 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260809 02:18:29.564157 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.553756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:43175 (local address 127.9.40.67:35977)
0809 02:18:29.553955 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.553995 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.554563 (+ 568us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.554790 (+ 227us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.554793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.555428 (+ 635us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:29.555666 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.555671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.556361 (+ 690us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.556362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.557381 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.557383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.557877 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.557878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.558283 (+ 405us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.558803 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:29.558811 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:29.560338 (+ 1527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:29.562311 (+ 1973us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.562320 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.562322 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.562698 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.562966 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:29.562967 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:29.562967 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:29.563106 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:29.563434 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:29.563442 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:29.563607 (+ 165us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.563719 (+ 112us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.563888 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":109,"threads_started":1}
I20260809 02:18:29.564626 28557 heartbeater.cc:347] Connected to a master server at 127.9.40.126:43175
I20260809 02:18:29.564770 28557 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:29.564955 28557 heartbeater.cc:511] Master 127.9.40.126:43175 requested a full tablet report, sending...
I20260809 02:18:29.565308 28057 ts_manager.cc:194] Registered new tserver with Master: 38ec2587588f484aabc5a2213d89f0b4 (127.9.40.67:36153)
I20260809 02:18:29.565668 28057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:35977
I20260809 02:18:29.573899 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:29.582423 28100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.575445 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54920 (local address 127.9.40.126:43175)
0809 02:18:29.575515 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.575522 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.575588 (+ 66us) server_negotiation.cc:407] Connection header received
0809 02:18:29.575648 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.575648 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.575659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.575706 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:29.576322 (+ 616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.576944 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.577715 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.577921 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.577989 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.579621 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:29.579622 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:29.579623 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:29.579632 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:29.581163 (+ 1531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.581576 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.581577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.581577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.581639 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:29.581854 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:29.581854 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:29.581855 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:29.582017 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:29.582113 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.582263 (+ 150us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.582333 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:18:29.584316 28057 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54920:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:29.586140 28057 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:29.593575 28573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.588990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42667 (local address 127.0.0.1:54214)
0809 02:18:29.589758 (+ 768us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.589769 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.589822 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.590094 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.590095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.590096 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:29.590134 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.590135 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.591011 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.591012 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.592274 (+ 1262us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.592277 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593177 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.593272 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593312 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.593389 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.593457 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":748,"mutex_wait_us":4,"thread_start_us":155,"threads_started":1}
I20260809 02:18:29.593636 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.589302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60752 (local address 127.9.40.66:43203)
0809 02:18:29.589513 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.589519 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.589532 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:18:29.589579 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.589581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.589599 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.589687 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:29.589912 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.591202 (+ 1290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.592516 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593243 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593330 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593389 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.593481 (+ 92us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.593526 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":93,"threads_started":1}
I20260809 02:18:29.593817 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.589272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46446 (local address 127.9.40.67:36153)
0809 02:18:29.589522 (+ 250us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.589526 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.589536 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:29.589579 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.589581 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.589599 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.589903 (+ 304us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:29.589982 (+ 79us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.591202 (+ 1220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.592565 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593274 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593619 (+ 345us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593683 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.593710 (+ 27us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.593745 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":100,"threads_started":1}
I20260809 02:18:29.594009 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.589511 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54214 (local address 127.9.40.65:42667)
0809 02:18:29.589685 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.589690 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.589880 (+ 190us) server_negotiation.cc:407] Connection header received
0809 02:18:29.589927 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.589929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.589947 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.590015 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:29.590212 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.590902 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.592374 (+ 1472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593069 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593163 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593231 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.593844 (+ 613us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.593892 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":134,"thread_start_us":99,"threads_started":1}
I20260809 02:18:29.593715 28572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.588990 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:36153 (local address 127.0.0.1:46446)
0809 02:18:29.589237 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.589279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.589353 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.589724 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.589727 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.589730 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:29.589811 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.589814 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.591320 (+ 1506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.591321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.592454 (+ 1133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.592456 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593370 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.593453 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593458 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.593502 (+ 44us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.593543 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":114,"threads_started":1}
I20260809 02:18:29.593782 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.588987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43203 (local address 127.0.0.1:60752)
0809 02:18:29.589236 (+ 249us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.589279 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.589351 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.589773 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.589774 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.589775 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:29.589823 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.589825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.591366 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.591367 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.592415 (+ 1048us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.592419 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.593337 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.593338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.593408 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.593411 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.593470 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.593503 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":50,"thread_start_us":124,"threads_started":1}
I20260809 02:18:29.595010 28334 tablet_service.cc:1467] Processing CreateTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 (DEFAULT_TABLE table=test-table [id=f092dd20ba2142abaff37ba80b8d2c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:29.595074 28486 tablet_service.cc:1467] Processing CreateTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 (DEFAULT_TABLE table=test-table [id=f092dd20ba2142abaff37ba80b8d2c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:29.595129 28192 tablet_service.cc:1467] Processing CreateTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 (DEFAULT_TABLE table=test-table [id=f092dd20ba2142abaff37ba80b8d2c75]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:29.595485 28192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72bef64abbf54c23a3eb3e640688d8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:29.595489 28486 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72bef64abbf54c23a3eb3e640688d8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:29.595487 28334 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 72bef64abbf54c23a3eb3e640688d8f9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:29.598359 28577 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Bootstrap starting.
I20260809 02:18:29.599154 28578 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Bootstrap starting.
I20260809 02:18:29.599287 28577 tablet_bootstrap.cc:654] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.599871 28578 tablet_bootstrap.cc:654] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.600008 28577 log.cc:824] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.600454 28578 log.cc:824] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.600831 28579 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Bootstrap starting.
I20260809 02:18:29.600908 28577 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: No bootstrap required, opened a new log
I20260809 02:18:29.600965 28577 ts_tablet_manager.cc:1397] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:18:29.601500 28579 tablet_bootstrap.cc:654] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:29.602119 28579 log.cc:824] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:29.602679 28578 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: No bootstrap required, opened a new log
I20260809 02:18:29.602751 28578 ts_tablet_manager.cc:1397] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:18:29.603241 28577 raft_consensus.cc:348] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.603432 28577 raft_consensus.cc:374] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.603466 28577 raft_consensus.cc:729] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 45d1750d3a284263bd78f4697ed7a5c2, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.603582 28577 consensus_queue.cc:260] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.603870 28577 ts_tablet_manager.cc:1428] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:29.604090 28262 heartbeater.cc:503] Master 127.9.40.126:43175 was elected leader, sending a full tablet report...
I20260809 02:18:29.604918 28578 raft_consensus.cc:348] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.605126 28579 tablet_bootstrap.cc:492] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: No bootstrap required, opened a new log
I20260809 02:18:29.605139 28578 raft_consensus.cc:374] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.605183 28578 raft_consensus.cc:729] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 38ec2587588f484aabc5a2213d89f0b4, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.605237 28579 ts_tablet_manager.cc:1397] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260809 02:18:29.605345 28578 consensus_queue.cc:260] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.605623 28578 ts_tablet_manager.cc:1428] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:29.605672 28557 heartbeater.cc:503] Master 127.9.40.126:43175 was elected leader, sending a full tablet report...
I20260809 02:18:29.607151 28579 raft_consensus.cc:348] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.607311 28579 raft_consensus.cc:374] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:29.607360 28579 raft_consensus.cc:729] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fa2f38075564115a65e5e4d0d2c7bf1, State: Initialized, Role: FOLLOWER
I20260809 02:18:29.607504 28579 consensus_queue.cc:260] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.607789 28579 ts_tablet_manager.cc:1428] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:29.607853 28407 heartbeater.cc:503] Master 127.9.40.126:43175 was elected leader, sending a full tablet report...
W20260809 02:18:29.640334 28408 tablet.cc:2369] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:29.732810 28263 tablet.cc:2369] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260809 02:18:29.803603 28558 tablet.cc:2369] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:29.828167 28584 raft_consensus.cc:482] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:29.828369 28584 raft_consensus.cc:504] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.828868 28584 leader_election.cc:288] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 45d1750d3a284263bd78f4697ed7a5c2 (127.9.40.65:42667), 1fa2f38075564115a65e5e4d0d2c7bf1 (127.9.40.66:43203)
I20260809 02:18:29.833294 28559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.829130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42667 (local address 127.9.40.67:54635)
0809 02:18:29.829198 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.829211 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.829263 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.829480 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.829482 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.829484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:29.829574 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.829575 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.830673 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.830674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.832135 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.832139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.832972 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832980 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.833103 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.833106 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.833160 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.833192 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":47,"mutex_wait_us":94}
I20260809 02:18:29.833343 28575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.829249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:53707 (local address 127.9.40.66:43203)
0809 02:18:29.829295 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.829297 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.829442 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:18:29.829544 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.829545 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.829557 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.829603 (+ 46us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:29.829776 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.830554 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832245 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.832891 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832964 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.832976 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.833219 (+ 243us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.833268 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:18:29.833340 28576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.829228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:54635 (local address 127.9.40.65:42667)
0809 02:18:29.829294 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.829297 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.829308 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:29.829344 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.829345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.829358 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.829429 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:29.829687 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.830552 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832246 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.832877 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832970 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.832979 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.833219 (+ 240us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.833267 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260809 02:18:29.833302 28586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.829164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43203 (local address 127.9.40.67:53707)
0809 02:18:29.829370 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.829379 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.829453 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.829627 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:29.829627 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:29.829630 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:29.829704 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.829705 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.830673 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.830674 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.832135 (+ 1461us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:29.832139 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.832972 (+ 833us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.832973 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:29.833063 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.833069 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:18:29.833138 (+ 69us) client_negotiation.cc:240] Negotiation successful
0809 02:18:29.833186 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192,"mutex_wait_us":89,"thread_start_us":146,"threads_started":1}
I20260809 02:18:29.833859 28355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72bef64abbf54c23a3eb3e640688d8f9" candidate_uuid: "38ec2587588f484aabc5a2213d89f0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" is_pre_election: true
I20260809 02:18:29.834050 28355 raft_consensus.cc:2454] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ec2587588f484aabc5a2213d89f0b4 in term 0.
I20260809 02:18:29.834017 28212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72bef64abbf54c23a3eb3e640688d8f9" candidate_uuid: "38ec2587588f484aabc5a2213d89f0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d1750d3a284263bd78f4697ed7a5c2" is_pre_election: true
I20260809 02:18:29.834205 28212 raft_consensus.cc:2454] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 38ec2587588f484aabc5a2213d89f0b4 in term 0.
I20260809 02:18:29.834311 28436 leader_election.cc:302] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa2f38075564115a65e5e4d0d2c7bf1, 38ec2587588f484aabc5a2213d89f0b4; no voters:
I20260809 02:18:29.834491 28584 raft_consensus.cc:2781] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:29.834542 28584 raft_consensus.cc:482] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:29.834564 28584 raft_consensus.cc:3037] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.835310 28584 raft_consensus.cc:504] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.835522 28584 leader_election.cc:288] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [CANDIDATE]: Term 1 election: Requested vote from peers 45d1750d3a284263bd78f4697ed7a5c2 (127.9.40.65:42667), 1fa2f38075564115a65e5e4d0d2c7bf1 (127.9.40.66:43203)
I20260809 02:18:29.835626 28355 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72bef64abbf54c23a3eb3e640688d8f9" candidate_uuid: "38ec2587588f484aabc5a2213d89f0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1"
I20260809 02:18:29.835685 28355 raft_consensus.cc:3037] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.835700 28212 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "72bef64abbf54c23a3eb3e640688d8f9" candidate_uuid: "38ec2587588f484aabc5a2213d89f0b4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "45d1750d3a284263bd78f4697ed7a5c2"
I20260809 02:18:29.835770 28212 raft_consensus.cc:3037] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:29.836377 28355 raft_consensus.cc:2454] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ec2587588f484aabc5a2213d89f0b4 in term 1.
I20260809 02:18:29.836540 28212 raft_consensus.cc:2454] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 38ec2587588f484aabc5a2213d89f0b4 in term 1.
I20260809 02:18:29.836570 28436 leader_election.cc:302] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1fa2f38075564115a65e5e4d0d2c7bf1, 38ec2587588f484aabc5a2213d89f0b4; no voters:
I20260809 02:18:29.836673 28584 raft_consensus.cc:2781] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:29.836876 28584 raft_consensus.cc:686] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 1 LEADER]: Becoming Leader. State: Replica: 38ec2587588f484aabc5a2213d89f0b4, State: Running, Role: LEADER
I20260809 02:18:29.836982 28584 consensus_queue.cc:237] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } }
I20260809 02:18:29.837817 28057 catalog_manager.cc:5495] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 reported cstate change: term changed from 0 to 1, leader changed from <none> to 38ec2587588f484aabc5a2213d89f0b4 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "38ec2587588f484aabc5a2213d89f0b4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "38ec2587588f484aabc5a2213d89f0b4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 36153 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:29.879943 28574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.877147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46450 (local address 127.9.40.67:36153)
0809 02:18:29.877220 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:29.877222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:29.877234 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:18:29.877300 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:29.877301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:29.877324 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:29.877394 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:29.877577 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.878153 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.879029 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:29.879246 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:29.879309 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:29.879489 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:29.879600 (+ 111us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:29.879671 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:29.879767 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:18:29.879788 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:18:29.882174 28355 raft_consensus.cc:1261] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 1 FOLLOWER]: Refusing update from remote peer 38ec2587588f484aabc5a2213d89f0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:29.882187 28212 raft_consensus.cc:1261] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 1 FOLLOWER]: Refusing update from remote peer 38ec2587588f484aabc5a2213d89f0b4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:29.882642 28584 consensus_queue.cc:1035] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "45d1750d3a284263bd78f4697ed7a5c2" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 42667 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:29.882851 28587 consensus_queue.cc:1035] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fa2f38075564115a65e5e4d0d2c7bf1" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 43203 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:29.885495 28593 mvcc.cc:205] Tried to move back new op lower bound from 7316446862873980928 to 7316446862692839424. Current Snapshot: MvccSnapshot[applied={T|T < 7316446862873980928}]
I20260809 02:18:29.886838 28590 mvcc.cc:205] Tried to move back new op lower bound from 7316446862873980928 to 7316446862692839424. Current Snapshot: MvccSnapshot[applied={T|T < 7316446862873980928 or (T in {7316446862873980928})}]
I20260809 02:18:29.888193 28056 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:29.888314 28056 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54920:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:29.889220 28056 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 18c2d5372ea24d6d90058210291e1454: Sending DeleteTablet for 3 replicas of tablet 72bef64abbf54c23a3eb3e640688d8f9
I20260809 02:18:29.889626 28334 tablet_service.cc:1514] Processing DeleteTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:29 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:60752
I20260809 02:18:29.889626 28192 tablet_service.cc:1514] Processing DeleteTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:29 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:54214
I20260809 02:18:29.889696 28486 tablet_service.cc:1514] Processing DeleteTablet for tablet 72bef64abbf54c23a3eb3e640688d8f9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:29 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:46446
I20260809 02:18:29.889890 28604 tablet_replica.cc:330] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2: stopping tablet replica
I20260809 02:18:29.890056 28604 raft_consensus.cc:2229] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:29.890229 28604 raft_consensus.cc:2258] T 72bef64abbf54c23a3eb3e640688d8f9 P 45d1750d3a284263bd78f4697ed7a5c2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:29.890252 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28122
I20260809 02:18:29.894515 28605 tablet_replica.cc:330] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: stopping tablet replica
I20260809 02:18:29.894666 28605 raft_consensus.cc:2229] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:29.894805 28605 raft_consensus.cc:2258] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:29.895308 28605 ts_tablet_manager.cc:1905] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:29.897370 28605 ts_tablet_manager.cc:1918] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:29.897475 28605 log.cc:1196] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/72bef64abbf54c23a3eb3e640688d8f9
I20260809 02:18:29.897871 28605 ts_tablet_manager.cc:1939] T 72bef64abbf54c23a3eb3e640688d8f9 P 38ec2587588f484aabc5a2213d89f0b4: Deleting consensus metadata
I20260809 02:18:29.898475 28040 catalog_manager.cc:4841] TS 38ec2587588f484aabc5a2213d89f0b4 (127.9.40.67:36153): tablet 72bef64abbf54c23a3eb3e640688d8f9 (table test-table [id=f092dd20ba2142abaff37ba80b8d2c75]) successfully deleted
W20260809 02:18:29.899398 28043 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:42667 (error 108)
I20260809 02:18:29.899770 28603 tablet_replica.cc:330] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: stopping tablet replica
I20260809 02:18:29.899950 28603 raft_consensus.cc:2229] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:29.900120 28603 raft_consensus.cc:2258] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260809 02:18:29.900429 28571 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:18:29.899892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:42667 (local address 127.0.0.1:54222)
0809 02:18:29.899947 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.899958 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:29.900004 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:29.900372 (+ 368us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42667: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:18:29.900586 28043 catalog_manager.cc:4584] TS 45d1750d3a284263bd78f4697ed7a5c2 (127.9.40.65:42667): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 72bef64abbf54c23a3eb3e640688d8f9: Network error: Client connection negotiation failed: client connection to 127.9.40.65:42667: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260809 02:18:29.900899 28603 ts_tablet_manager.cc:1905] T 72bef64abbf54c23a3eb3e640688d8f9 P 1fa2f38075564115a65e5e4d0d2c7bf1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:29.900981 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28267
I20260809 02:18:29.908840 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28414
I20260809 02:18:29.908996 28571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:29.908816 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:43203 (local address 127.0.0.1:60764)
0809 02:18:29.908868 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:29.908947 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:43203: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 02:18:29.915954 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28021
2026-08-09T02:18:29Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2317 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-09T02:18:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:29Z Disabled control of system clock
I20260809 02:18:29.950961 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37055
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:37055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:30.037107 28612 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:30.037335 28612 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:30.037357 28612 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:30.038429 28612 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:30.038467 28612 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:30.038480 28612 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:30.038493 28612 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:30.038506 28612 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:30.039794 28612 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:37055
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:37055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28612
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:30.040005 28612 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:30.040287 28612 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:30.045950 28620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.046077 28612 server_base.cc:1034] running on GCE node
W20260809 02:18:30.045950 28617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:30.046022 28618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.046461 28612 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:30.046775 28612 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:30.047950 28612 hybrid_clock.cc:648] HybridClock initialized: now 1786241910047929 us; error 46 us; skew 500 ppm
I20260809 02:18:30.049407 28612 webserver.cc:460] Webserver started at http://127.9.40.126:44001/ using document root <none> and password file <none>
I20260809 02:18:30.049714 28612 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:30.049788 28612 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:30.049938 28612 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:30.051090 28612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "7aa25e38b2ba4b3e835439559f2fcab9"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "eb8cde6fedea36a72ce43582a92917e3"
server_key_iv: "2db2994c20bfb6c7d4005275348f152f"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.051453 28612 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "7aa25e38b2ba4b3e835439559f2fcab9"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "eb8cde6fedea36a72ce43582a92917e3"
server_key_iv: "2db2994c20bfb6c7d4005275348f152f"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.052860 28612 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:30.053781 28627 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.054039 28612 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:30.054150 28612 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "7aa25e38b2ba4b3e835439559f2fcab9"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "eb8cde6fedea36a72ce43582a92917e3"
server_key_iv: "2db2994c20bfb6c7d4005275348f152f"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.054239 28612 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:30.073294 28612 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:30.073632 28612 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:30.073817 28612 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:30.078293 28612 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:37055
I20260809 02:18:30.078337 28683 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:37055 every 8 connection(s)
I20260809 02:18:30.078711 28612 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:30.079777 28684 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:30.082475 28684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: Bootstrap starting.
I20260809 02:18:30.083128 28684 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:30.083455 28684 log.cc:824] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:30.084216 28684 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: No bootstrap required, opened a new log
I20260809 02:18:30.086092 28684 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } }
I20260809 02:18:30.086243 28684 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:30.086268 28684 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7aa25e38b2ba4b3e835439559f2fcab9, State: Initialized, Role: FOLLOWER
I20260809 02:18:30.086484 28684 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } }
I20260809 02:18:30.086563 28684 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:30.086589 28684 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:30.086623 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28612
I20260809 02:18:30.086660 28684 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:30.086732 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:30.086923 9377 external_mini_cluster.cc:1428] Setting key c1a6f445c7c01c8d06ce1fa883033dc9
I20260809 02:18:30.087419 28684 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } }
I20260809 02:18:30.087555 28684 leader_election.cc:302] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7aa25e38b2ba4b3e835439559f2fcab9; no voters:
I20260809 02:18:30.087754 28684 leader_election.cc:288] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:30.087790 28689 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:30.087915 28689 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [term 1 LEADER]: Becoming Leader. State: Replica: 7aa25e38b2ba4b3e835439559f2fcab9, State: Running, Role: LEADER
I20260809 02:18:30.088037 28684 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:30.088002 28689 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } }
I20260809 02:18:30.088498 28690 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } } }
I20260809 02:18:30.088594 28690 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:30.088860 28691 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7aa25e38b2ba4b3e835439559f2fcab9. Latest consensus state: current_term: 1 leader_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7aa25e38b2ba4b3e835439559f2fcab9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 37055 } } }
I20260809 02:18:30.088955 28691 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:30.089088 28696 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:30.089561 28696 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:30.093279 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.087479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53142 (local address 127.9.40.126:37055)
0809 02:18:30.087787 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.087796 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.087829 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:18:30.087950 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.087958 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.088027 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.088402 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:30.088510 (+ 108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.089953 (+ 1443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.090990 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.091619 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.091757 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.092468 (+ 711us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:30.092472 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:30.092473 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:30.092682 (+ 209us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:30.092761 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:30.092779 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:30.092853 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.092970 (+ 117us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.093060 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":147,"threads_started":1}
I20260809 02:18:30.093842 28696 catalog_manager.cc:1347] Generated new cluster ID: 164d930eff914c5ba927ca24ec00a5c7
I20260809 02:18:30.093899 28696 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:30.104018 28696 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:30.104835 28696 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:30.111474 28696 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: Generated new TSK 0
I20260809 02:18:30.111719 28696 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:30.113677 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--builtin_ntp_servers=127.9.40.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:30.208194 28710 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:30.208489 28710 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:30.208516 28710 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:30.209798 28710 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:30.209843 28710 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:30.209868 28710 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:30.211369 28710 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28710
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:30.211658 28710 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:30.212003 28710 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:30.218881 28716 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.218952 28710 server_base.cc:1034] running on GCE node
W20260809 02:18:30.218881 28717 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:30.218875 28719 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.219379 28710 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:30.219756 28710 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:30.220932 28710 hybrid_clock.cc:648] HybridClock initialized: now 1786241910220914 us; error 32 us; skew 500 ppm
I20260809 02:18:30.222641 28710 webserver.cc:460] Webserver started at http://127.9.40.65:37713/ using document root <none> and password file <none>
I20260809 02:18:30.222968 28710 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:30.223029 28710 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:30.223184 28710 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:30.224435 28710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "188497d911fe4f4fa413c8d7c20809e0"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "98981601b964f8b24fb5cb24a1350777"
server_key_iv: "2a7ed869c61ee7e483129deafb0e94dd"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.224890 28710 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "188497d911fe4f4fa413c8d7c20809e0"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "98981601b964f8b24fb5cb24a1350777"
server_key_iv: "2a7ed869c61ee7e483129deafb0e94dd"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.226473 28710 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:30.227373 28726 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.227587 28710 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:30.227687 28710 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "188497d911fe4f4fa413c8d7c20809e0"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "98981601b964f8b24fb5cb24a1350777"
server_key_iv: "2a7ed869c61ee7e483129deafb0e94dd"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.227774 28710 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:30.243170 28710 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:30.243547 28710 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:30.243721 28710 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:30.244019 28710 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:30.244534 28710 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:30.244578 28710 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.244603 28710 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:30.244617 28710 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.251577 28710 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:39803
I20260809 02:18:30.251622 28850 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:39803 every 8 connection(s)
I20260809 02:18:30.252018 28710 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:30.258047 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.252949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44637 (local address 127.9.40.126:37055)
0809 02:18:30.253024 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.253027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.253278 (+ 251us) server_negotiation.cc:407] Connection header received
0809 02:18:30.253789 (+ 511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.253790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.253802 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.253877 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:30.254639 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.255186 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.256272 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.256488 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.256556 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.257752 (+ 1196us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:30.257753 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:30.257753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:30.257759 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:30.257776 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:30.257780 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:30.257840 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.257930 (+ 90us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.257970 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:18:30.258360 28853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.252915 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37055 (local address 127.9.40.65:44637)
0809 02:18:30.253132 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.253179 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.253699 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.253936 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.253940 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.253949 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:30.254529 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.254534 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.255281 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.255282 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.256182 (+ 900us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.256184 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.256584 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.256585 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.256709 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.257599 (+ 890us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:30.257599 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:30.257633 (+ 34us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:30.257661 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:30.257845 (+ 184us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:30.257847 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:30.257850 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.257933 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.258061 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":110,"threads_started":1}
I20260809 02:18:30.259096 28851 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37055
I20260809 02:18:30.259209 28851 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:30.259444 28851 heartbeater.cc:511] Master 127.9.40.126:37055 requested a full tablet report, sending...
I20260809 02:18:30.260030 28647 ts_manager.cc:194] Registered new tserver with Master: 188497d911fe4f4fa413c8d7c20809e0 (127.9.40.65:39803)
I20260809 02:18:30.260567 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28710
I20260809 02:18:30.260684 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:30.260955 9377 external_mini_cluster.cc:1428] Setting key b2b23c2b934ed298659fe10e8b1f2d5d
I20260809 02:18:30.261101 28647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:44637
I20260809 02:18:30.262176 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--builtin_ntp_servers=127.9.40.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:30.347334 28854 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:30.347604 28854 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:30.347641 28854 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:30.348846 28854 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:30.348930 28854 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:30.348973 28854 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:30.350375 28854 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28854
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:30.350687 28854 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:30.351017 28854 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:30.356384 28860 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:30.356401 28859 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.356521 28854 server_base.cc:1034] running on GCE node
W20260809 02:18:30.356392 28863 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.356936 28854 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:30.357251 28854 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:30.358440 28854 hybrid_clock.cc:648] HybridClock initialized: now 1786241910358413 us; error 41 us; skew 500 ppm
I20260809 02:18:30.359844 28854 webserver.cc:460] Webserver started at http://127.9.40.66:40229/ using document root <none> and password file <none>
I20260809 02:18:30.360147 28854 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:30.360226 28854 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:30.360388 28854 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:30.361542 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "f4f2543a1f3b49469695cc12e23afc1a"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "4fcf044d7246548541b52402076e8e10"
server_key_iv: "85197dd67a23869763c08ec6623cc14c"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.361946 28854 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "f4f2543a1f3b49469695cc12e23afc1a"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "4fcf044d7246548541b52402076e8e10"
server_key_iv: "85197dd67a23869763c08ec6623cc14c"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.363381 28854 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:18:30.364238 28869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.364477 28854 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:30.364584 28854 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "f4f2543a1f3b49469695cc12e23afc1a"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "4fcf044d7246548541b52402076e8e10"
server_key_iv: "85197dd67a23869763c08ec6623cc14c"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.364665 28854 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:30.380224 28854 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:30.380632 28854 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:30.380887 28854 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:30.381238 28854 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:30.381621 28854 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:30.381680 28854 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.381722 28854 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:30.381752 28854 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.387804 28854 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39123
I20260809 02:18:30.387850 28993 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39123 every 8 connection(s)
I20260809 02:18:30.388332 28854 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:30.394253 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.389190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:43951 (local address 127.9.40.126:37055)
0809 02:18:30.389244 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.389246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.389577 (+ 331us) server_negotiation.cc:407] Connection header received
0809 02:18:30.390209 (+ 632us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.390211 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.390224 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.390286 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:30.391188 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.391709 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.392740 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.392956 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.393039 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.393955 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:30.393957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:30.393957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:30.393963 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:30.393978 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:30.393981 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:30.394029 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.394140 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.394171 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:18:30.394505 28996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.389216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37055 (local address 127.9.40.66:43951)
0809 02:18:30.389445 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.389484 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.390084 (+ 600us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.390360 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.390363 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.390372 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:30.391054 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.391060 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.391816 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.391817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.392612 (+ 795us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.392615 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.393049 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.393051 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.393188 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.393794 (+ 606us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:30.393795 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:30.393824 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:30.393846 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:30.394029 (+ 183us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:30.394034 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:30.394037 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.394140 (+ 103us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.394246 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":88,"threads_started":1}
I20260809 02:18:30.395010 28994 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37055
I20260809 02:18:30.395120 28994 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:30.395309 28994 heartbeater.cc:511] Master 127.9.40.126:37055 requested a full tablet report, sending...
I20260809 02:18:30.395741 28647 ts_manager.cc:194] Registered new tserver with Master: f4f2543a1f3b49469695cc12e23afc1a (127.9.40.66:39123)
I20260809 02:18:30.396136 28647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:43951
I20260809 02:18:30.398490 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28854
I20260809 02:18:30.398634 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:30.398859 9377 external_mini_cluster.cc:1428] Setting key 65e52e67586c7eaf6b9f0e282d44a43a
I20260809 02:18:30.399806 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--builtin_ntp_servers=127.9.40.84:42787
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:30.483865 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:30.484068 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:30.484090 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:30.485297 28997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:30.485338 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:30.485363 28997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:30.486725 28997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:42787
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:37055
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:30.486982 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:30.487303 28997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:30.492877 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:30.492826 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:30.492847 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:30.493081 28997 server_base.cc:1034] running on GCE node
I20260809 02:18:30.493268 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:30.493484 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:30.494635 28997 hybrid_clock.cc:648] HybridClock initialized: now 1786241910494634 us; error 40 us; skew 500 ppm
I20260809 02:18:30.495915 28997 webserver.cc:460] Webserver started at http://127.9.40.67:44833/ using document root <none> and password file <none>
I20260809 02:18:30.496186 28997 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:30.496234 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:30.496387 28997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:30.497611 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "1bd657e4560d4c51b6e5edf7ae4f656c"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "6aec9673f616faf587b3ae94754893d4"
server_key_iv: "f20a0b40691ea69b73c16bc58ff5bcea"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.497989 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "1bd657e4560d4c51b6e5edf7ae4f656c"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "6aec9673f616faf587b3ae94754893d4"
server_key_iv: "f20a0b40691ea69b73c16bc58ff5bcea"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.499451 28997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:18:30.500313 29013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.500550 28997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:30.500783 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "1bd657e4560d4c51b6e5edf7ae4f656c"
format_stamp: "Formatted at 2026-08-09 02:18:30 on dist-test-slave-qg60"
server_key: "6aec9673f616faf587b3ae94754893d4"
server_key_iv: "f20a0b40691ea69b73c16bc58ff5bcea"
server_key_version: "encryptionkey@0"
I20260809 02:18:30.500878 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:30.511942 28997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:30.512298 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:30.512458 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:30.512720 28997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:30.513110 28997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:30.513168 28997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.513214 28997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:30.513242 28997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:30.519290 28997 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:39431
I20260809 02:18:30.519344 29136 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:39431 every 8 connection(s)
I20260809 02:18:30.519711 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:30.525000 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.520540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:36173 (local address 127.9.40.126:37055)
0809 02:18:30.520605 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.520607 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.520906 (+ 299us) server_negotiation.cc:407] Connection header received
0809 02:18:30.521392 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.521393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.521403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.521473 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:30.522111 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.522622 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.523606 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.523797 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.523852 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.524678 (+ 826us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:30.524680 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:30.524680 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:30.524684 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:30.524697 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:30.524701 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:30.524817 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.524914 (+ 97us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.524939 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:18:30.525259 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.520521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:37055 (local address 127.9.40.67:36173)
0809 02:18:30.520778 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.520826 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.521306 (+ 480us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.521525 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.521528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.521536 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:30.522029 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.522033 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.522709 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.522710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.523524 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.523527 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.523878 (+ 351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.523879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.523989 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.524524 (+ 535us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:30.524525 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:30.524553 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:30.524580 (+ 27us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:30.524831 (+ 251us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:30.524838 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:30.524840 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.524904 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.524986 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":130,"threads_started":1}
I20260809 02:18:30.525566 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 28997
I20260809 02:18:30.525686 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:30.525839 29137 heartbeater.cc:347] Connected to a master server at 127.9.40.126:37055
I20260809 02:18:30.525920 29137 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:30.525926 9377 external_mini_cluster.cc:1428] Setting key 40c6bc59dc3cd0dfad9984be5f62b9fe
I20260809 02:18:30.526062 29137 heartbeater.cc:511] Master 127.9.40.126:37055 requested a full tablet report, sending...
I20260809 02:18:30.526645 28647 ts_manager.cc:194] Registered new tserver with Master: 1bd657e4560d4c51b6e5edf7ae4f656c (127.9.40.67:39431)
I20260809 02:18:30.527160 28647 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:36173
I20260809 02:18:30.527436 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:30.532922 28688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.529135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53150 (local address 127.9.40.126:37055)
0809 02:18:30.529175 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.529177 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.529381 (+ 204us) server_negotiation.cc:407] Connection header received
0809 02:18:30.529439 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.529440 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.529453 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.529520 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:30.529690 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.530390 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.531430 (+ 1040us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.531673 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.531733 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.532551 (+ 818us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:30.532552 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:30.532552 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:30.532556 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:30.532569 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:30.532572 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:30.532675 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.532794 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.532826 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:18:30.534929 28647 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53150:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:30.536455 28647 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:30.543470 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.539246 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57138 (local address 127.9.40.65:39803)
0809 02:18:30.539426 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.539430 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.539547 (+ 117us) server_negotiation.cc:407] Connection header received
0809 02:18:30.539592 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.539595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.539610 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.539664 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:30.539908 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.540694 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.542305 (+ 1611us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.543072 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.543153 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.543249 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.543351 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.543382 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":73,"threads_started":1}
I20260809 02:18:30.543481 29147 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.539140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39803 (local address 127.0.0.1:57138)
0809 02:18:30.539440 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.539448 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.539495 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.539784 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.539785 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.539787 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:30.539825 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.539826 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.540814 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.540815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.542201 (+ 1386us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.542205 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.543160 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.543161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.543247 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.543264 (+ 17us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.543341 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.543397 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":286,"mutex_wait_us":25,"thread_start_us":77,"threads_started":1}
I20260809 02:18:30.543941 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.538886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41130 (local address 127.9.40.66:39123)
0809 02:18:30.539119 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.539124 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.539338 (+ 214us) server_negotiation.cc:407] Connection header received
0809 02:18:30.539465 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.539468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.539485 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.539539 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:30.539808 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.540989 (+ 1181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.542306 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.543081 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.543153 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.543249 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.543840 (+ 591us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.543873 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":101,"threads_started":1}
I20260809 02:18:30.543921 29150 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.538784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39123 (local address 127.0.0.1:41130)
0809 02:18:30.539273 (+ 489us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.539282 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.539363 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.539633 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.539634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.539636 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:30.539704 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.539706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.541076 (+ 1370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.541077 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.542202 (+ 1125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.542205 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.543690 (+ 1485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.543691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.543750 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.543754 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.543815 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.543854 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":471,"mutex_wait_us":3,"thread_start_us":91,"threads_started":1}
I20260809 02:18:30.544807 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.538784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:39431 (local address 127.0.0.1:60798)
0809 02:18:30.539189 (+ 405us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.539249 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.539336 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.539617 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.539620 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.539623 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:30.539697 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.539700 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.542309 (+ 2609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.542310 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.543144 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.543147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.544565 (+ 1418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.544566 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.544627 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.544630 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.544682 (+ 52us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.544718 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"spinlock_wait_cycles":768,"thread_start_us":96,"threads_started":1}
I20260809 02:18:30.544872 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.538983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60798 (local address 127.9.40.67:39431)
0809 02:18:30.539216 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.539220 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.539351 (+ 131us) server_negotiation.cc:407] Connection header received
0809 02:18:30.539417 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.539420 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.539438 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.539522 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:30.539771 (+ 249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.540989 (+ 1218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.543244 (+ 2255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.543987 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.544035 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.544145 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.544722 (+ 577us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.544791 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":82,"threads_started":1}
I20260809 02:18:30.544934 28920 tablet_service.cc:1467] Processing CreateTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 (DEFAULT_TABLE table=test-table [id=019a76745f7945eda02d6dc09281462d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:30.544970 28779 tablet_service.cc:1467] Processing CreateTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 (DEFAULT_TABLE table=test-table [id=019a76745f7945eda02d6dc09281462d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:30.545285 28920 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4204d256ec54cef8d95a6c43fab51a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:30.545316 28779 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4204d256ec54cef8d95a6c43fab51a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:30.547780 29153 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: Bootstrap starting.
I20260809 02:18:30.547778 29154 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Bootstrap starting.
I20260809 02:18:30.548520 29154 tablet_bootstrap.cc:654] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:30.548525 29153 tablet_bootstrap.cc:654] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:30.548604 29063 tablet_service.cc:1467] Processing CreateTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 (DEFAULT_TABLE table=test-table [id=019a76745f7945eda02d6dc09281462d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:30.548888 29154 log.cc:824] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:30.548944 29153 log.cc:824] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:30.549032 29063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4204d256ec54cef8d95a6c43fab51a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:30.549690 29154 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: No bootstrap required, opened a new log
I20260809 02:18:30.549777 29154 ts_tablet_manager.cc:1397] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:30.551760 29157 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Bootstrap starting.
I20260809 02:18:30.551656 29154 raft_consensus.cc:348] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.551829 29154 raft_consensus.cc:374] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:30.551867 29154 raft_consensus.cc:729] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f4f2543a1f3b49469695cc12e23afc1a, State: Initialized, Role: FOLLOWER
I20260809 02:18:30.551990 29154 consensus_queue.cc:260] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.552299 29154 ts_tablet_manager.cc:1428] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Time spent starting tablet: real 0.002s user 0.003s sys 0.000s
I20260809 02:18:30.552354 28994 heartbeater.cc:503] Master 127.9.40.126:37055 was elected leader, sending a full tablet report...
I20260809 02:18:30.552438 29157 tablet_bootstrap.cc:654] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:30.552819 29157 log.cc:824] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:30.552934 29153 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: No bootstrap required, opened a new log
I20260809 02:18:30.553013 29153 ts_tablet_manager.cc:1397] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260809 02:18:30.553601 29157 tablet_bootstrap.cc:492] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: No bootstrap required, opened a new log
I20260809 02:18:30.553721 29157 ts_tablet_manager.cc:1397] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:30.555855 29153 raft_consensus.cc:348] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.556030 29153 raft_consensus.cc:374] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:30.556066 29153 raft_consensus.cc:729] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 188497d911fe4f4fa413c8d7c20809e0, State: Initialized, Role: FOLLOWER
I20260809 02:18:30.556198 29153 consensus_queue.cc:260] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.556213 29157 raft_consensus.cc:348] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.556361 29157 raft_consensus.cc:374] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:30.556412 29157 raft_consensus.cc:729] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1bd657e4560d4c51b6e5edf7ae4f656c, State: Initialized, Role: FOLLOWER
I20260809 02:18:30.556481 29153 ts_tablet_manager.cc:1428] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260809 02:18:30.556520 29157 consensus_queue.cc:260] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.556531 28851 heartbeater.cc:503] Master 127.9.40.126:37055 was elected leader, sending a full tablet report...
I20260809 02:18:30.556867 29137 heartbeater.cc:503] Master 127.9.40.126:37055 was elected leader, sending a full tablet report...
I20260809 02:18:30.557097 29157 ts_tablet_manager.cc:1428] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
W20260809 02:18:30.638936 28995 tablet.cc:2369] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:30.709642 29162 raft_consensus.cc:482] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:30.709771 29162 raft_consensus.cc:504] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.710170 29162 leader_election.cc:288] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f4f2543a1f3b49469695cc12e23afc1a (127.9.40.66:39123), 188497d911fe4f4fa413c8d7c20809e0 (127.9.40.65:39803)
I20260809 02:18:30.714051 29151 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.710784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:48387 (local address 127.9.40.65:39803)
0809 02:18:30.710860 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.710862 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.710873 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:30.710901 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.710902 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.710914 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.710969 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:30.711158 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.711829 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713099 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.713672 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713744 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.713755 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.713931 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.713976 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260809 02:18:30.714072 29163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.710539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39803 (local address 127.9.40.67:48387)
0809 02:18:30.710743 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.710756 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.710806 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.710979 (+ 173us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.710980 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.710981 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:30.711082 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.711084 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.711898 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.711899 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.713019 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.713021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.713762 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.713847 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.713850 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.713911 (+ 61us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.713969 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"mutex_wait_us":1,"thread_start_us":124,"threads_started":1}
I20260809 02:18:30.714116 29148 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.710476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:42465 (local address 127.9.40.66:39123)
0809 02:18:30.710520 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.710522 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.710538 (+ 16us) server_negotiation.cc:407] Connection header received
0809 02:18:30.710587 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.710587 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.710596 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.710676 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:30.710894 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.711624 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713088 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.713660 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713744 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.713755 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.713931 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.714039 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:18:30.714502 29139 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.710384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39123 (local address 127.9.40.67:42465)
0809 02:18:30.710446 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.710459 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:30.710508 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:30.710683 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:30.710684 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:30.710686 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:30.710778 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.710780 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.711703 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.711703 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.712950 (+ 1247us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:30.712953 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.713760 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.713761 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:30.713842 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.713846 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:30.713911 (+ 65us) client_negotiation.cc:240] Negotiation successful
0809 02:18:30.713972 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36,"mutex_wait_us":23}
I20260809 02:18:30.714501 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4204d256ec54cef8d95a6c43fab51a1" candidate_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "188497d911fe4f4fa413c8d7c20809e0" is_pre_election: true
I20260809 02:18:30.714687 28802 raft_consensus.cc:2454] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd657e4560d4c51b6e5edf7ae4f656c in term 0.
I20260809 02:18:30.714852 28941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4204d256ec54cef8d95a6c43fab51a1" candidate_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f2543a1f3b49469695cc12e23afc1a" is_pre_election: true
I20260809 02:18:30.714991 28941 raft_consensus.cc:2454] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1bd657e4560d4c51b6e5edf7ae4f656c in term 0.
I20260809 02:18:30.714963 29015 leader_election.cc:302] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 188497d911fe4f4fa413c8d7c20809e0, 1bd657e4560d4c51b6e5edf7ae4f656c; no voters:
I20260809 02:18:30.715103 29162 raft_consensus.cc:2781] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:30.715166 29162 raft_consensus.cc:482] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:30.715191 29162 raft_consensus.cc:3037] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:30.715895 29162 raft_consensus.cc:504] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.716035 29162 leader_election.cc:288] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [CANDIDATE]: Term 1 election: Requested vote from peers f4f2543a1f3b49469695cc12e23afc1a (127.9.40.66:39123), 188497d911fe4f4fa413c8d7c20809e0 (127.9.40.65:39803)
I20260809 02:18:30.716171 28941 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4204d256ec54cef8d95a6c43fab51a1" candidate_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f4f2543a1f3b49469695cc12e23afc1a"
I20260809 02:18:30.716238 28802 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4204d256ec54cef8d95a6c43fab51a1" candidate_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "188497d911fe4f4fa413c8d7c20809e0"
I20260809 02:18:30.716257 28941 raft_consensus.cc:3037] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:30.716326 28802 raft_consensus.cc:3037] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:30.717310 28802 raft_consensus.cc:2454] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd657e4560d4c51b6e5edf7ae4f656c in term 1.
I20260809 02:18:30.717310 28941 raft_consensus.cc:2454] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1bd657e4560d4c51b6e5edf7ae4f656c in term 1.
I20260809 02:18:30.717515 29015 leader_election.cc:302] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1bd657e4560d4c51b6e5edf7ae4f656c, f4f2543a1f3b49469695cc12e23afc1a; no voters:
I20260809 02:18:30.717674 29162 raft_consensus.cc:2781] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:30.717847 29162 raft_consensus.cc:686] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 1 LEADER]: Becoming Leader. State: Replica: 1bd657e4560d4c51b6e5edf7ae4f656c, State: Running, Role: LEADER
I20260809 02:18:30.717960 29162 consensus_queue.cc:237] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } }
I20260809 02:18:30.718940 28647 catalog_manager.cc:5495] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c reported cstate change: term changed from 0 to 1, leader changed from <none> to 1bd657e4560d4c51b6e5edf7ae4f656c (127.9.40.67). New cstate: current_term: 1 leader_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1bd657e4560d4c51b6e5edf7ae4f656c" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 39431 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 } health_report { overall_health: UNKNOWN } } }
I20260809 02:18:30.729429 29149 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.726854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60814 (local address 127.9.40.67:39431)
0809 02:18:30.726916 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:30.726917 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:30.726927 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:30.726968 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:30.726968 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:30.726984 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:30.727048 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:30.727148 (+ 100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.727689 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.728461 (+ 772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:30.728654 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:30.728711 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:30.728925 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:30.729027 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:30.729134 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:30.729299 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:18:30.729318 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:18:30.731495 28802 raft_consensus.cc:1261] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 1 FOLLOWER]: Refusing update from remote peer 1bd657e4560d4c51b6e5edf7ae4f656c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:30.731549 28941 raft_consensus.cc:1261] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 1 FOLLOWER]: Refusing update from remote peer 1bd657e4560d4c51b6e5edf7ae4f656c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:30.731995 29164 consensus_queue.cc:1035] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [LEADER]: Connected to new peer: Peer: permanent_uuid: "188497d911fe4f4fa413c8d7c20809e0" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 39803 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:30.732157 29162 consensus_queue.cc:1035] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [LEADER]: Connected to new peer: Peer: permanent_uuid: "f4f2543a1f3b49469695cc12e23afc1a" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:30.734962 29169 mvcc.cc:205] Tried to move back new op lower bound from 7316446866352881664 to 7316446866301669376. Current Snapshot: MvccSnapshot[applied={T|T < 7316446866352881664}]
I20260809 02:18:30.737727 28644 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:30.737833 28644 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53150:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:30.738708 28644 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7aa25e38b2ba4b3e835439559f2fcab9: Sending DeleteTablet for 3 replicas of tablet e4204d256ec54cef8d95a6c43fab51a1
I20260809 02:18:30.739097 29063 tablet_service.cc:1514] Processing DeleteTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:30 UTC) from {username='slave'} at 127.0.0.1:60798
I20260809 02:18:30.739123 28920 tablet_service.cc:1514] Processing DeleteTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:30 UTC) from {username='slave'} at 127.0.0.1:41130
I20260809 02:18:30.739231 28779 tablet_service.cc:1514] Processing DeleteTablet for tablet e4204d256ec54cef8d95a6c43fab51a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:30 UTC) from {username='slave'} at 127.0.0.1:57138
I20260809 02:18:30.739480 29180 tablet_replica.cc:330] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0: stopping tablet replica
I20260809 02:18:30.739668 29180 raft_consensus.cc:2229] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:30.739675 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28710
I20260809 02:18:30.739853 29180 raft_consensus.cc:2258] T e4204d256ec54cef8d95a6c43fab51a1 P 188497d911fe4f4fa413c8d7c20809e0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:30.757251 29178 tablet_replica.cc:330] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: stopping tablet replica
I20260809 02:18:30.757539 29177 tablet_replica.cc:330] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: stopping tablet replica
I20260809 02:18:30.757702 29177 raft_consensus.cc:2229] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:30.757862 29177 raft_consensus.cc:2258] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:30.758107 29178 raft_consensus.cc:2229] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:30.758445 29178 raft_consensus.cc:2258] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:30.758563 29177 ts_tablet_manager.cc:1905] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:30.758945 29178 ts_tablet_manager.cc:1905] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:18:30.760694 28629 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:39803 (error 108)
I20260809 02:18:30.760959 29177 ts_tablet_manager.cc:1918] T e4204d256ec54cef8d95a6c43fab51a1 P f4f2543a1f3b49469695cc12e23afc1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:30.760982 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28854
I20260809 02:18:30.761085 29178 ts_tablet_manager.cc:1918] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:30.761161 29178 log.cc:1196] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/e4204d256ec54cef8d95a6c43fab51a1
I20260809 02:18:30.761409 29178 ts_tablet_manager.cc:1939] T e4204d256ec54cef8d95a6c43fab51a1 P 1bd657e4560d4c51b6e5edf7ae4f656c: Deleting consensus metadata
I20260809 02:18:30.762189 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.762031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:39803 (local address 127.0.0.1:57144)
0809 02:18:30.762093 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.762148 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39803: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:18:30.762302 28629 catalog_manager.cc:4584] TS 188497d911fe4f4fa413c8d7c20809e0 (127.9.40.65:39803): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e4204d256ec54cef8d95a6c43fab51a1: Network error: Client connection negotiation failed: client connection to 127.9.40.65:39803: connect: Connection refused (error 111)
I20260809 02:18:30.762607 28631 catalog_manager.cc:4841] TS 1bd657e4560d4c51b6e5edf7ae4f656c (127.9.40.67:39431): tablet e4204d256ec54cef8d95a6c43fab51a1 (table test-table [id=019a76745f7945eda02d6dc09281462d]) successfully deleted
I20260809 02:18:30.773854 29146 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:30.773689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39123 (local address 127.0.0.1:41146)
0809 02:18:30.773765 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:30.773804 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:39123: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260809 02:18:30.773903 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28997
I20260809 02:18:30.779596 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 28612
2026-08-09T02:18:30Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (861 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:30 dist-test-slave-qg60 krb5kdc[29184](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:30 dist-test-slave-qg60 krb5kdc[29184](info): set up 2 sockets
Aug 09 02:18:30 dist-test-slave-qg60 krb5kdc[29184](info): commencing operation
krb5kdc: starting...
W20260809 02:18:31.877388 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:31 dist-test-slave-qg60 krb5kdc[29184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:32.009968 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46245
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:46245
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:32.094285 29197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:32.094515 29197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:32.094539 29197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:32.095829 29197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:32.095880 29197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:32.095894 29197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:32.095908 29197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:32.095920 29197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:32.097551 29197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:46245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:46245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:32.097829 29197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:32.098076 29197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:32.103250 29205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.103308 29203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.103308 29202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.103590 29197 server_base.cc:1034] running on GCE node
I20260809 02:18:32.103843 29197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:32.104162 29197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:32.105333 29197 hybrid_clock.cc:648] HybridClock initialized: now 1786241912105317 us; error 51 us; skew 500 ppm
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:32.115166 29197 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:32.115759 29197 webserver.cc:460] Webserver started at http://127.9.40.126:39393/ using document root <none> and password file <none>
I20260809 02:18:32.116026 29197 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:32.116104 29197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:32.116263 29197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:32.117439 29197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "e2411d8184b4402dbf5874e45106b273"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "a799d63cb478736df94c66c954c8ac3f"
server_key_iv: "ef4dcff623466e5e3f9b7b93a4e497db"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.117823 29197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "e2411d8184b4402dbf5874e45106b273"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "a799d63cb478736df94c66c954c8ac3f"
server_key_iv: "ef4dcff623466e5e3f9b7b93a4e497db"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.119273 29197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:32.120051 29215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.120250 29197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:32.120335 29197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "e2411d8184b4402dbf5874e45106b273"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "a799d63cb478736df94c66c954c8ac3f"
server_key_iv: "ef4dcff623466e5e3f9b7b93a4e497db"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.120428 29197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:32.132877 29197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:32.134953 29197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:32.135107 29197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:32.139530 29197 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:46245
I20260809 02:18:32.139588 29270 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:46245 every 8 connection(s)
I20260809 02:18:32.139953 29197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:32.140911 29272 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:32.143324 29272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: Bootstrap starting.
I20260809 02:18:32.143965 29272 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:32.144282 29272 log.cc:824] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:32.145113 29272 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: No bootstrap required, opened a new log
I20260809 02:18:32.145908 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29197
I20260809 02:18:32.146039 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:32.146243 9377 external_mini_cluster.cc:1428] Setting key 8db3fc169e525947d3664ce37ee28615
I20260809 02:18:32.147302 29272 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } }
I20260809 02:18:32.147490 29272 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:32.147536 29272 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2411d8184b4402dbf5874e45106b273, State: Initialized, Role: FOLLOWER
I20260809 02:18:32.147703 29272 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } }
I20260809 02:18:32.147797 29272 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:32.147835 29272 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:32.147889 29272 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:32.149022 29272 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } }
I20260809 02:18:32.149116 29272 leader_election.cc:302] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2411d8184b4402dbf5874e45106b273; no voters:
I20260809 02:18:32.149346 29272 leader_election.cc:288] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:32.149379 29278 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:32.149549 29278 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [term 1 LEADER]: Becoming Leader. State: Replica: e2411d8184b4402dbf5874e45106b273, State: Running, Role: LEADER
I20260809 02:18:32.149636 29278 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } }
I20260809 02:18:32.149636 29272 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:32.151405 29278 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2411d8184b4402dbf5874e45106b273. Latest consensus state: current_term: 1 leader_uuid: "e2411d8184b4402dbf5874e45106b273" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } } }
I20260809 02:18:32.151499 29278 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:32.151648 29280 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2411d8184b4402dbf5874e45106b273" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2411d8184b4402dbf5874e45106b273" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 46245 } } }
I20260809 02:18:32.151722 29294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:32.151772 29280 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:32.152251 29294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241911, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:32.154036 29294 catalog_manager.cc:1347] Generated new cluster ID: ca9649724bfe4727a49caa2a888432d6
I20260809 02:18:32.154105 29294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:32.158373 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.146828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40058 (local address 127.9.40.126:46245)
0809 02:18:32.147024 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.147038 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.147065 (+ 27us) server_negotiation.cc:407] Connection header received
0809 02:18:32.147258 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.147264 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.147316 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.147430 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.148294 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.149458 (+ 1164us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.150332 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.151064 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.151180 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.154379 (+ 3199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.154382 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.154383 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.154395 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.156342 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.156760 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.156763 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.156763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.156888 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.157079 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.157080 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.157080 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.157370 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.157592 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.158043 (+ 451us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.158173 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":104,"threads_started":1}
I20260809 02:18:32.168462 29294 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:32.169162 29294 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:32.171360 29294 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: Generated new TSK 0
I20260809 02:18:32.171546 29294 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:18:32.205197 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:32.300915 29303 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:32.301162 29303 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:32.301204 29303 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:32.302398 29303 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:32.302472 29303 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:32.302517 29303 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:32.304013 29303 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29303
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:32.304363 29303 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:32.304615 29303 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:32.309836 29309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.309813 29308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.310127 29303 server_base.cc:1034] running on GCE node
W20260809 02:18:32.309816 29311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.310412 29303 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:32.310673 29303 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:32.311841 29303 hybrid_clock.cc:648] HybridClock initialized: now 1786241912311818 us; error 42 us; skew 500 ppm
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:32.314428 29303 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:18:32.314896 29303 webserver.cc:460] Webserver started at http://127.9.40.65:38703/ using document root <none> and password file <none>
I20260809 02:18:32.315136 29303 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:32.315205 29303 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:32.315354 29303 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:32.316455 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "9990ce39dcb24d4399e780ca1eb66cf7"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "edbbbcbd2844cd0983d00a2934510298"
server_key_iv: "6e51f55b09d263b1da22f25406ec236a"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.316896 29303 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "9990ce39dcb24d4399e780ca1eb66cf7"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "edbbbcbd2844cd0983d00a2934510298"
server_key_iv: "6e51f55b09d263b1da22f25406ec236a"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.318252 29303 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:32.318993 29319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.319219 29303 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:32.319305 29303 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "9990ce39dcb24d4399e780ca1eb66cf7"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "edbbbcbd2844cd0983d00a2934510298"
server_key_iv: "6e51f55b09d263b1da22f25406ec236a"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.319360 29303 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:32.339155 29303 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:32.341132 29303 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:32.341248 29303 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:32.341463 29303 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:32.341796 29303 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:32.341868 29303 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.341912 29303 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:32.341936 29303 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.347815 29303 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37165
I20260809 02:18:32.347898 29442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37165 every 8 connection(s)
I20260809 02:18:32.348228 29303 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:32.352133 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29303
I20260809 02:18:32.352267 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:32.352480 9377 external_mini_cluster.cc:1428] Setting key c7919697026ee723a9fa20031e7b28b2
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:32.359177 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.349064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:55777 (local address 127.9.40.126:46245)
0809 02:18:32.349151 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.349153 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.349373 (+ 220us) server_negotiation.cc:407] Connection header received
0809 02:18:32.349971 (+ 598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.349972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.349991 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.350043 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.351098 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.351630 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.353114 (+ 1484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.353331 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.353422 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.355789 (+ 2367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.355791 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.355792 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.355806 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.357453 (+ 1647us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.358033 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.358035 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.358035 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.358105 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.358387 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.358388 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.358389 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.358589 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.358674 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.358930 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.359032 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:18:32.359195 29445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.349035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46245 (local address 127.9.40.65:55777)
0809 02:18:32.349216 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.349283 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.349859 (+ 576us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.350143 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.350148 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.350785 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:32.350990 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.350994 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.351742 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.351743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.353014 (+ 1271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.353017 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.353437 (+ 420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.353438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.353570 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.354132 (+ 562us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:32.354141 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:32.355686 (+ 1545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:32.357557 (+ 1871us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.357566 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.357569 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.357896 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.358201 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.358202 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.358202 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.358311 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.358667 (+ 356us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:32.358672 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:32.358793 (+ 121us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.358879 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.359010 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":85,"threads_started":1}
I20260809 02:18:32.359793 29443 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46245
I20260809 02:18:32.359911 29443 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:32.360136 29443 heartbeater.cc:511] Master 127.9.40.126:46245 requested a full tablet report, sending...
I20260809 02:18:32.360829 29232 ts_manager.cc:194] Registered new tserver with Master: 9990ce39dcb24d4399e780ca1eb66cf7 (127.9.40.65:37165)
I20260809 02:18:32.361505 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:55777
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:18:32.384363 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:32.470010 29449 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:32.470234 29449 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:32.470259 29449 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:32.471464 29449 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:32.471515 29449 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:32.471544 29449 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:32.473158 29449 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29449
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:32.473423 29449 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:32.473680 29449 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:32.478933 29459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.478955 29456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.478950 29455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.479197 29449 server_base.cc:1034] running on GCE node
I20260809 02:18:32.479389 29449 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:32.479650 29449 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:32.480805 29449 hybrid_clock.cc:648] HybridClock initialized: now 1786241912480783 us; error 27 us; skew 500 ppm
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:32.483299 29449 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:18:32.483762 29449 webserver.cc:460] Webserver started at http://127.9.40.66:42087/ using document root <none> and password file <none>
I20260809 02:18:32.483992 29449 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:32.484062 29449 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:32.484225 29449 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:32.485389 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "246312729f2a4c1db3a6e18063b19ac0"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "5ab79beb9fd63d159786fabf0b56a35c"
server_key_iv: "e07fdf60e361352802638a698b0beb04"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.485772 29449 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "246312729f2a4c1db3a6e18063b19ac0"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "5ab79beb9fd63d159786fabf0b56a35c"
server_key_iv: "e07fdf60e361352802638a698b0beb04"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.487123 29449 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:32.487848 29466 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.488024 29449 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:32.488134 29449 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "246312729f2a4c1db3a6e18063b19ac0"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "5ab79beb9fd63d159786fabf0b56a35c"
server_key_iv: "e07fdf60e361352802638a698b0beb04"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.488221 29449 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:32.503566 29449 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:32.505717 29449 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:32.505899 29449 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:32.506206 29449 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:32.506705 29449 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:32.506771 29449 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.506829 29449 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:32.506856 29449 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.513185 29449 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35351
I20260809 02:18:32.513275 29592 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35351 every 8 connection(s)
I20260809 02:18:32.513615 29449 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:32.520390 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29449
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:32.520493 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:32.520813 9377 external_mini_cluster.cc:1428] Setting key 709db1c1b5fc173fbdacd095217c8976
I20260809 02:18:32.524399 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.514591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:36413 (local address 127.9.40.126:46245)
0809 02:18:32.514714 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.514716 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.514882 (+ 166us) server_negotiation.cc:407] Connection header received
0809 02:18:32.515543 (+ 661us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.515544 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.515565 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.515620 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.516712 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.517308 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.518343 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.518539 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.518624 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.521089 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.521115 (+ 26us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.521115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.521125 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.522820 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.523313 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.523315 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.523315 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.523395 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.523735 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.523736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.523736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.523849 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.523903 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.524256 (+ 353us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.524333 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260809 02:18:32.524793 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.514504 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46245 (local address 127.9.40.66:36413)
0809 02:18:32.514734 (+ 230us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.514784 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.515410 (+ 626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.515701 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.515708 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.516389 (+ 681us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:32.516610 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.516615 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.517420 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.517421 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.518234 (+ 813us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.518235 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.518632 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.518634 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.518761 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.519338 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:32.519346 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:32.520963 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:32.522921 (+ 1958us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.522929 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.522931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.523196 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.523479 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.523479 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.523480 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.523666 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.523918 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:32.523925 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:32.524107 (+ 182us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.524234 (+ 127us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.524460 (+ 226us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":100,"threads_started":1}
I20260809 02:18:32.525303 29593 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46245
I20260809 02:18:32.525415 29593 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:32.525626 29593 heartbeater.cc:511] Master 127.9.40.126:46245 requested a full tablet report, sending...
I20260809 02:18:32.526006 29232 ts_manager.cc:194] Registered new tserver with Master: 246312729f2a4c1db3a6e18063b19ac0 (127.9.40.66:35351)
I20260809 02:18:32.526472 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:36413
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:18:32.551226 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:40069
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:32.635135 29599 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:32.635347 29599 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:32.635371 29599 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:32.636471 29599 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:32.636507 29599 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:32.636531 29599 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:32.637954 29599 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40069
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:46245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29599
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:32.638227 29599 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:32.638511 29599 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:32.643849 29605 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:32.643857 29604 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.643966 29599 server_base.cc:1034] running on GCE node
W20260809 02:18:32.643857 29608 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:32.644345 29599 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:32.644564 29599 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:32.645722 29599 hybrid_clock.cc:648] HybridClock initialized: now 1786241912645676 us; error 60 us; skew 500 ppm
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:32.648409 29599 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:18:32.649062 29599 webserver.cc:460] Webserver started at http://127.9.40.67:38983/ using document root <none> and password file <none>
I20260809 02:18:32.649351 29599 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:32.649417 29599 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:32.649565 29599 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:32.650933 29599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "aa633134f504477b86df89ac834b625e"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "052dfa9d4541cc5392c34fc550e65d7f"
server_key_iv: "d4673a951c2ea3d388354acfe260aaaf"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.651373 29599 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "aa633134f504477b86df89ac834b625e"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "052dfa9d4541cc5392c34fc550e65d7f"
server_key_iv: "d4673a951c2ea3d388354acfe260aaaf"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.653143 29599 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260809 02:18:32.654018 29615 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.654230 29599 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:32.654345 29599 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "aa633134f504477b86df89ac834b625e"
format_stamp: "Formatted at 2026-08-09 02:18:32 on dist-test-slave-qg60"
server_key: "052dfa9d4541cc5392c34fc550e65d7f"
server_key_iv: "d4673a951c2ea3d388354acfe260aaaf"
server_key_version: "encryptionkey@0"
I20260809 02:18:32.654428 29599 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:32.665625 29599 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:32.667608 29599 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:32.667783 29599 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:32.668035 29599 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:32.668359 29599 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:32.668414 29599 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.668462 29599 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:32.668485 29599 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:32.674825 29599 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44069
I20260809 02:18:32.674870 29742 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44069 every 8 connection(s)
I20260809 02:18:32.675269 29599 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:32.676623 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29599
I20260809 02:18:32.676734 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:32.676926 9377 external_mini_cluster.cc:1428] Setting key 2f07d0b76f6be679b8e965ef7acc7755
Aug 09 02:18:32 dist-test-slave-qg60 krb5kdc[29184](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241912, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:32.684902 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.676049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:45539 (local address 127.9.40.126:46245)
0809 02:18:32.676141 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.676144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.676335 (+ 191us) server_negotiation.cc:407] Connection header received
0809 02:18:32.676987 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.676989 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.677002 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.677049 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.678205 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.678711 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.679728 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.679936 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.680021 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.682277 (+ 2256us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.682278 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.682278 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.682284 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.683518 (+ 1234us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.683894 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.683895 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.683895 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.683945 (+ 50us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.684237 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.684238 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.684238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.684341 (+ 103us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.684390 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.684692 (+ 302us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.684808 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:32.685122 29745 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.676012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:46245 (local address 127.9.40.67:45539)
0809 02:18:32.676206 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.676250 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.676861 (+ 611us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.677137 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.677141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.677888 (+ 747us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:32.678096 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.678101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.678809 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.678810 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.679619 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.679621 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.680033 (+ 412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.680034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.680164 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.680681 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:32.680688 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:32.682192 (+ 1504us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:32.683604 (+ 1412us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.683610 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.683612 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.683827 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.684033 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:32.684033 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:32.684034 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:32.684179 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:32.684397 (+ 218us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:32.684402 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:32.684538 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.684651 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.684832 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":99,"threads_started":1}
I20260809 02:18:32.685590 29743 heartbeater.cc:347] Connected to a master server at 127.9.40.126:46245
I20260809 02:18:32.685681 29743 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:32.685832 29743 heartbeater.cc:511] Master 127.9.40.126:46245 requested a full tablet report, sending...
I20260809 02:18:32.686241 29232 ts_manager.cc:194] Registered new tserver with Master: aa633134f504477b86df89ac834b625e (127.9.40.67:44069)
I20260809 02:18:32.686640 29232 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:45539
I20260809 02:18:32.688652 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:32.696390 29277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.690117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40070 (local address 127.9.40.126:46245)
0809 02:18:32.690179 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.690181 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.690274 (+ 93us) server_negotiation.cc:407] Connection header received
0809 02:18:32.690309 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.690310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.690318 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.690372 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:32.690881 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.691414 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.692219 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.692396 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.692452 (+ 56us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.693909 (+ 1457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:32.693910 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:32.693910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:32.693917 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:32.695150 (+ 1233us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.695533 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.695534 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.695534 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.695611 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:32.695860 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:32.695861 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:32.695861 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:32.695997 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:32.696083 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.696269 (+ 186us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.696322 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:18:32.698166 29232 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40070:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:32.699903 29232 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:32.707023 29757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35351 (local address 127.0.0.1:37678)
0809 02:18:32.702758 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.702816 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.702868 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.703166 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.703169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.703172 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:32.703261 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.703263 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.704586 (+ 1323us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.704587 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.705839 (+ 1252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.705842 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.706684 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.706685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.706780 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.706794 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.706866 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.706918 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":68,"threads_started":1}
I20260809 02:18:32.707048 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37678 (local address 127.9.40.66:35351)
0809 02:18:32.702969 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.702974 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.702984 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:32.703026 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.703027 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.703042 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.703129 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:32.703330 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.704466 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.705931 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.706591 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.706674 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.706737 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.706895 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.706939 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":103,"threads_started":1}
I20260809 02:18:32.707667 29758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44069 (local address 127.0.0.1:50334)
0809 02:18:32.703548 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.703557 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.703618 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.703869 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.703870 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.703872 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:32.703909 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.703911 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.705087 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.705088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.706205 (+ 1117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.706207 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.707382 (+ 1175us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.707383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.707460 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.707466 (+ 6us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.707525 (+ 59us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.707583 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":767,"mutex_wait_us":11,"thread_start_us":72,"threads_started":1}
I20260809 02:18:32.707703 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50334 (local address 127.9.40.67:44069)
0809 02:18:32.703094 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.703147 (+ 53us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.703636 (+ 489us) server_negotiation.cc:407] Connection header received
0809 02:18:32.703685 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.703687 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.703702 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.703787 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:32.703988 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.704991 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.706455 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.707225 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.707295 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.707345 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.707565 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.707610 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":96,"threads_started":1}
I20260809 02:18:32.708101 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37165 (local address 127.0.0.1:33922)
0809 02:18:32.702506 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.702539 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.702624 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.703668 (+ 1044us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.703668 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.703669 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:32.703711 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.703712 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.705944 (+ 2232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.705944 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.706920 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.706922 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.707860 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.707861 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.707921 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.707925 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.707992 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.708030 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
I20260809 02:18:32.708106 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.702516 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33922 (local address 127.9.40.65:37165)
0809 02:18:32.703294 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.703299 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.703308 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:18:32.703350 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.703352 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.703366 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.703441 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:32.703785 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.704472 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.707013 (+ 2541us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.707764 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.707850 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.707914 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.708014 (+ 100us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.708047 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":737,"thread_start_us":141,"threads_started":1}
I20260809 02:18:32.708169 29520 tablet_service.cc:1467] Processing CreateTablet for tablet e27b8060e5c9487d877a9a7180295615 (DEFAULT_TABLE table=test-table [id=eec512c3c5f64c3d84e44898ed21a3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:32.708525 29520 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27b8060e5c9487d877a9a7180295615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:32.709774 29668 tablet_service.cc:1467] Processing CreateTablet for tablet e27b8060e5c9487d877a9a7180295615 (DEFAULT_TABLE table=test-table [id=eec512c3c5f64c3d84e44898ed21a3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:32.709801 29372 tablet_service.cc:1467] Processing CreateTablet for tablet e27b8060e5c9487d877a9a7180295615 (DEFAULT_TABLE table=test-table [id=eec512c3c5f64c3d84e44898ed21a3b1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:32.710103 29668 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27b8060e5c9487d877a9a7180295615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:32.710142 29372 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e27b8060e5c9487d877a9a7180295615. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:32.712510 29762 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Bootstrap starting.
I20260809 02:18:32.712812 29763 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Bootstrap starting.
I20260809 02:18:32.713258 29762 tablet_bootstrap.cc:654] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:32.713496 29763 tablet_bootstrap.cc:654] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:32.713821 29764 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Bootstrap starting.
I20260809 02:18:32.713865 29762 log.cc:824] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:32.714076 29763 log.cc:824] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:32.714640 29764 tablet_bootstrap.cc:654] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:32.714704 29762 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: No bootstrap required, opened a new log
I20260809 02:18:32.714777 29762 ts_tablet_manager.cc:1397] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:32.714929 29763 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: No bootstrap required, opened a new log
I20260809 02:18:32.715019 29763 ts_tablet_manager.cc:1397] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:32.715229 29764 log.cc:824] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:32.715953 29764 tablet_bootstrap.cc:492] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: No bootstrap required, opened a new log
I20260809 02:18:32.716068 29764 ts_tablet_manager.cc:1397] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:32.716933 29762 raft_consensus.cc:348] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.717126 29762 raft_consensus.cc:374] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:32.717052 29763 raft_consensus.cc:348] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.717187 29762 raft_consensus.cc:729] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 246312729f2a4c1db3a6e18063b19ac0, State: Initialized, Role: FOLLOWER
I20260809 02:18:32.717233 29763 raft_consensus.cc:374] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:32.717278 29763 raft_consensus.cc:729] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aa633134f504477b86df89ac834b625e, State: Initialized, Role: FOLLOWER
I20260809 02:18:32.717317 29762 consensus_queue.cc:260] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.717396 29763 consensus_queue.cc:260] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.717628 29762 ts_tablet_manager.cc:1428] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:32.717703 29763 ts_tablet_manager.cc:1428] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:32.717753 29743 heartbeater.cc:503] Master 127.9.40.126:46245 was elected leader, sending a full tablet report...
I20260809 02:18:32.717772 29593 heartbeater.cc:503] Master 127.9.40.126:46245 was elected leader, sending a full tablet report...
I20260809 02:18:32.718712 29764 raft_consensus.cc:348] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.718880 29764 raft_consensus.cc:374] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:32.718927 29764 raft_consensus.cc:729] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9990ce39dcb24d4399e780ca1eb66cf7, State: Initialized, Role: FOLLOWER
I20260809 02:18:32.719049 29764 consensus_queue.cc:260] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.719328 29764 ts_tablet_manager.cc:1428] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:32.719414 29443 heartbeater.cc:503] Master 127.9.40.126:46245 was elected leader, sending a full tablet report...
W20260809 02:18:32.764235 29595 tablet.cc:2369] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:32.818065 29769 raft_consensus.cc:482] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:32.818286 29769 raft_consensus.cc:504] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.818775 29769 leader_election.cc:288] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9990ce39dcb24d4399e780ca1eb66cf7 (127.9.40.65:37165), aa633134f504477b86df89ac834b625e (127.9.40.67:44069)
I20260809 02:18:32.822546 29596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.818997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37165 (local address 127.9.40.66:49211)
0809 02:18:32.819071 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.819087 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.819141 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.819361 (+ 220us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.819362 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.819364 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:32.819459 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.819462 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.820229 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.820230 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.821362 (+ 1132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.821365 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.822216 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.822217 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.822312 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.822316 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.822380 (+ 64us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.822425 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260809 02:18:32.822551 29761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.819291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:37409 (local address 127.9.40.67:44069)
0809 02:18:32.819347 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.819349 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.819459 (+ 110us) server_negotiation.cc:407] Connection header received
0809 02:18:32.819492 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.819493 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.819507 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.819570 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:32.819765 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.820403 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.821559 (+ 1156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.822209 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.822291 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.822300 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.822426 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.822472 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260809 02:18:32.822551 29756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.819076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:49211 (local address 127.9.40.65:37165)
0809 02:18:32.819125 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.819128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.819141 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:18:32.819242 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.819243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.819256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.819338 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:32.819580 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.820129 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.821464 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.822098 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.822187 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.822199 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.822435 (+ 236us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.822481 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:18:32.822579 29772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.819148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44069 (local address 127.9.40.66:37409)
0809 02:18:32.819340 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.819353 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:32.819421 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:32.819601 (+ 180us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:32.819602 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:32.819604 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:32.819675 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.819676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.820536 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.820537 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.821421 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:32.821425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.822285 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.822286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:32.822347 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.822349 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:18:32.822403 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:18:32.822430 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":128,"threads_started":1}
I20260809 02:18:32.823019 29393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27b8060e5c9487d877a9a7180295615" candidate_uuid: "246312729f2a4c1db3a6e18063b19ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" is_pre_election: true
I20260809 02:18:32.823292 29393 raft_consensus.cc:2454] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246312729f2a4c1db3a6e18063b19ac0 in term 0.
I20260809 02:18:32.823243 29694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27b8060e5c9487d877a9a7180295615" candidate_uuid: "246312729f2a4c1db3a6e18063b19ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa633134f504477b86df89ac834b625e" is_pre_election: true
I20260809 02:18:32.823385 29694 raft_consensus.cc:2454] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 246312729f2a4c1db3a6e18063b19ac0 in term 0.
I20260809 02:18:32.823560 29467 leader_election.cc:302] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246312729f2a4c1db3a6e18063b19ac0, 9990ce39dcb24d4399e780ca1eb66cf7; no voters:
I20260809 02:18:32.823740 29769 raft_consensus.cc:2781] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:32.823825 29769 raft_consensus.cc:482] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:32.823868 29769 raft_consensus.cc:3037] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:32.824678 29769 raft_consensus.cc:504] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.824873 29769 leader_election.cc:288] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [CANDIDATE]: Term 1 election: Requested vote from peers 9990ce39dcb24d4399e780ca1eb66cf7 (127.9.40.65:37165), aa633134f504477b86df89ac834b625e (127.9.40.67:44069)
I20260809 02:18:32.825234 29694 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27b8060e5c9487d877a9a7180295615" candidate_uuid: "246312729f2a4c1db3a6e18063b19ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aa633134f504477b86df89ac834b625e"
I20260809 02:18:32.825315 29694 raft_consensus.cc:3037] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:32.826072 29694 raft_consensus.cc:2454] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246312729f2a4c1db3a6e18063b19ac0 in term 1.
I20260809 02:18:32.826201 29393 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e27b8060e5c9487d877a9a7180295615" candidate_uuid: "246312729f2a4c1db3a6e18063b19ac0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9990ce39dcb24d4399e780ca1eb66cf7"
I20260809 02:18:32.826256 29467 leader_election.cc:302] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 246312729f2a4c1db3a6e18063b19ac0, aa633134f504477b86df89ac834b625e; no voters:
I20260809 02:18:32.826290 29393 raft_consensus.cc:3037] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:32.826340 29769 raft_consensus.cc:2781] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:32.826498 29769 raft_consensus.cc:686] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 1 LEADER]: Becoming Leader. State: Replica: 246312729f2a4c1db3a6e18063b19ac0, State: Running, Role: LEADER
I20260809 02:18:32.826717 29769 consensus_queue.cc:237] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } }
I20260809 02:18:32.827248 29393 raft_consensus.cc:2454] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 246312729f2a4c1db3a6e18063b19ac0 in term 1.
I20260809 02:18:32.827683 29232 catalog_manager.cc:5495] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 246312729f2a4c1db3a6e18063b19ac0 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "246312729f2a4c1db3a6e18063b19ac0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "246312729f2a4c1db3a6e18063b19ac0" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35351 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 } health_report { overall_health: UNKNOWN } } }
W20260809 02:18:32.848973 29444 tablet.cc:2369] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:32.855305 29759 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.852776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37694 (local address 127.9.40.66:35351)
0809 02:18:32.852829 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:32.852831 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:32.852841 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:32.852888 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:32.852890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:32.852922 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:32.853006 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:32.853195 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.853681 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.854456 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:32.854659 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:32.854741 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:32.854896 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:32.854990 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:32.855157 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:32.855188 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 02:18:32.855197 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260809 02:18:32.857390 29694 raft_consensus.cc:1261] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 1 FOLLOWER]: Refusing update from remote peer 246312729f2a4c1db3a6e18063b19ac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:32.857434 29393 raft_consensus.cc:1261] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 1 FOLLOWER]: Refusing update from remote peer 246312729f2a4c1db3a6e18063b19ac0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:32.857788 29773 consensus_queue.cc:1035] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aa633134f504477b86df89ac834b625e" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44069 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:32.857947 29769 consensus_queue.cc:1035] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9990ce39dcb24d4399e780ca1eb66cf7" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37165 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:32.860293 29777 mvcc.cc:205] Tried to move back new op lower bound from 7316446875060523008 to 7316446874938949632. Current Snapshot: MvccSnapshot[applied={T|T < 7316446875060523008}]
I20260809 02:18:32.860674 29778 mvcc.cc:205] Tried to move back new op lower bound from 7316446875060523008 to 7316446874938949632. Current Snapshot: MvccSnapshot[applied={T|T < 7316446875060523008}]
I20260809 02:18:32.861042 29779 mvcc.cc:205] Tried to move back new op lower bound from 7316446875060523008 to 7316446874938949632. Current Snapshot: MvccSnapshot[applied={T|T < 7316446875060523008 or (T in {7316446875060523008})}]
I20260809 02:18:32.863121 29231 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:32.863236 29231 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40070:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:32.863989 29231 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e2411d8184b4402dbf5874e45106b273: Sending DeleteTablet for 3 replicas of tablet e27b8060e5c9487d877a9a7180295615
I20260809 02:18:32.864343 29372 tablet_service.cc:1514] Processing DeleteTablet for tablet e27b8060e5c9487d877a9a7180295615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:33922
I20260809 02:18:32.864372 29520 tablet_service.cc:1514] Processing DeleteTablet for tablet e27b8060e5c9487d877a9a7180295615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37678
I20260809 02:18:32.864411 29668 tablet_service.cc:1514] Processing DeleteTablet for tablet e27b8060e5c9487d877a9a7180295615 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:32 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:50334
I20260809 02:18:32.864629 29786 tablet_replica.cc:330] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7: stopping tablet replica
I20260809 02:18:32.864766 29786 raft_consensus.cc:2229] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:32.864760 29787 tablet_replica.cc:330] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: stopping tablet replica
I20260809 02:18:32.864859 29787 raft_consensus.cc:2229] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:32.864853 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29303
I20260809 02:18:32.864876 29786 raft_consensus.cc:2258] T e27b8060e5c9487d877a9a7180295615 P 9990ce39dcb24d4399e780ca1eb66cf7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:32.866015 29787 raft_consensus.cc:2258] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:32.872979 29787 ts_tablet_manager.cc:1905] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:32.875061 29787 ts_tablet_manager.cc:1918] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:32.875159 29787 log.cc:1196] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/e27b8060e5c9487d877a9a7180295615
I20260809 02:18:32.875445 29787 ts_tablet_manager.cc:1939] T e27b8060e5c9487d877a9a7180295615 P 246312729f2a4c1db3a6e18063b19ac0: Deleting consensus metadata
I20260809 02:18:32.876005 29216 catalog_manager.cc:4841] TS 246312729f2a4c1db3a6e18063b19ac0 (127.9.40.66:35351): tablet e27b8060e5c9487d877a9a7180295615 (table test-table [id=eec512c3c5f64c3d84e44898ed21a3b1]) successfully deleted
I20260809 02:18:32.876890 29788 tablet_replica.cc:330] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: stopping tablet replica
I20260809 02:18:32.877029 29788 raft_consensus.cc:2229] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:32.877152 29788 raft_consensus.cc:2258] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:32.877665 29788 ts_tablet_manager.cc:1905] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:18:32.878049 29216 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37165 (error 108)
I20260809 02:18:32.878361 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29449
I20260809 02:18:32.878732 29755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:32.878595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37165 (local address 127.0.0.1:33938)
0809 02:18:32.878660 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:32.878695 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37165: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260809 02:18:32.879464 29216 catalog_manager.cc:4584] TS 9990ce39dcb24d4399e780ca1eb66cf7 (127.9.40.65:37165): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e27b8060e5c9487d877a9a7180295615: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37165: connect: Connection refused (error 111)
I20260809 02:18:32.879715 29788 ts_tablet_manager.cc:1918] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:32.879796 29788 log.cc:1196] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/e27b8060e5c9487d877a9a7180295615
I20260809 02:18:32.880065 29788 ts_tablet_manager.cc:1939] T e27b8060e5c9487d877a9a7180295615 P aa633134f504477b86df89ac834b625e: Deleting consensus metadata
I20260809 02:18:32.880510 29216 catalog_manager.cc:4841] TS aa633134f504477b86df89ac834b625e (127.9.40.67:44069): tablet e27b8060e5c9487d877a9a7180295615 (table test-table [id=eec512c3c5f64c3d84e44898ed21a3b1]) successfully deleted
I20260809 02:18:32.885193 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29599
I20260809 02:18:32.892402 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29197
2026-08-09T02:18:32Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2112 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-09T02:18:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:32Z Disabled control of system clock
I20260809 02:18:32.924420 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40003
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:33.016963 29793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.017225 29793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.017251 29793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.018498 29793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:33.018540 29793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.018553 29793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.018565 29793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:33.018577 29793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:33.020071 29793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40003
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:33.020310 29793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.020581 29793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.025892 29802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.025945 29799 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.025902 29800 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.026208 29793 server_base.cc:1034] running on GCE node
I20260809 02:18:33.026439 29793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.026754 29793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.027932 29793 hybrid_clock.cc:648] HybridClock initialized: now 1786241913027904 us; error 46 us; skew 500 ppm
I20260809 02:18:33.029637 29793 webserver.cc:460] Webserver started at http://127.9.40.126:40785/ using document root <none> and password file <none>
I20260809 02:18:33.029874 29793 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.029922 29793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.030063 29793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.031391 29793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "366843ffad1b4a029b6789e2b7a2a8a3"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "17e3d9c4d025c8e23d1f8b2a12bf95d2"
server_key_iv: "2f510ddb20d43df5a560547fc13ff62d"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.031756 29793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "366843ffad1b4a029b6789e2b7a2a8a3"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "17e3d9c4d025c8e23d1f8b2a12bf95d2"
server_key_iv: "2f510ddb20d43df5a560547fc13ff62d"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.033317 29793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:18:33.034255 29808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.034492 29793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:33.034616 29793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "366843ffad1b4a029b6789e2b7a2a8a3"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "17e3d9c4d025c8e23d1f8b2a12bf95d2"
server_key_iv: "2f510ddb20d43df5a560547fc13ff62d"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.034704 29793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.047657 29793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.048032 29793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.048206 29793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.052966 29793 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:40003
I20260809 02:18:33.053045 29870 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:40003 every 8 connection(s)
I20260809 02:18:33.053457 29793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:33.054515 29871 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:33.056934 29871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: Bootstrap starting.
I20260809 02:18:33.057619 29871 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:33.057941 29871 log.cc:824] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:33.058727 29871 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: No bootstrap required, opened a new log
I20260809 02:18:33.059851 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29793
I20260809 02:18:33.059969 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:33.060206 9377 external_mini_cluster.cc:1428] Setting key 3dc9f3eefa0fe2c81735a1003895bff8
I20260809 02:18:33.060967 29871 raft_consensus.cc:348] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } }
I20260809 02:18:33.061163 29871 raft_consensus.cc:374] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:33.061213 29871 raft_consensus.cc:729] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 366843ffad1b4a029b6789e2b7a2a8a3, State: Initialized, Role: FOLLOWER
I20260809 02:18:33.061384 29871 consensus_queue.cc:260] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } }
I20260809 02:18:33.061479 29871 raft_consensus.cc:388] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:33.061520 29871 raft_consensus.cc:482] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:33.061575 29871 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:33.062150 29871 raft_consensus.cc:504] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } }
I20260809 02:18:33.062274 29871 leader_election.cc:302] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 366843ffad1b4a029b6789e2b7a2a8a3; no voters:
I20260809 02:18:33.062501 29871 leader_election.cc:288] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:33.062536 29876 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:33.062654 29876 raft_consensus.cc:686] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [term 1 LEADER]: Becoming Leader. State: Replica: 366843ffad1b4a029b6789e2b7a2a8a3, State: Running, Role: LEADER
I20260809 02:18:33.062739 29876 consensus_queue.cc:237] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } }
I20260809 02:18:33.062803 29871 sys_catalog.cc:564] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:33.063100 29877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } } }
I20260809 02:18:33.063191 29877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:33.063344 29878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 366843ffad1b4a029b6789e2b7a2a8a3. Latest consensus state: current_term: 1 leader_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "366843ffad1b4a029b6789e2b7a2a8a3" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40003 } } }
I20260809 02:18:33.063400 29878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:33.063711 29882 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:33.064217 29882 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:33.066344 29882 catalog_manager.cc:1347] Generated new cluster ID: cba87be8f56e4f8cbb6b7c4b6cc3e534
I20260809 02:18:33.066408 29882 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:33.066994 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.060874 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59588 (local address 127.9.40.126:40003)
0809 02:18:33.061126 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.061136 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.061172 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:18:33.061319 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.061327 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.061404 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.061538 (+ 134us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.061786 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.063408 (+ 1622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.064609 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.065259 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.065400 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.066195 (+ 795us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.066199 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.066199 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:33.066424 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.066472 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:33.066488 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.066559 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.066697 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.066788 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164,"thread_start_us":104,"threads_started":1}
I20260809 02:18:33.074925 29882 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:33.075692 29882 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:33.080438 29882 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: Generated new TSK 0
I20260809 02:18:33.080639 29882 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:33.084450 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--builtin_ntp_servers=127.9.40.84:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:33.168797 29897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.169054 29897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.169092 29897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.170384 29897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.170451 29897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.170509 29897 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:33.171972 29897 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:33.172257 29897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.172600 29897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.178270 29906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.178246 29902 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.178225 29903 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.179078 29897 server_base.cc:1034] running on GCE node
I20260809 02:18:33.179308 29897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.179582 29897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.180774 29897 hybrid_clock.cc:648] HybridClock initialized: now 1786241913180755 us; error 32 us; skew 500 ppm
I20260809 02:18:33.182400 29897 webserver.cc:460] Webserver started at http://127.9.40.65:39899/ using document root <none> and password file <none>
I20260809 02:18:33.182734 29897 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.182798 29897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.182930 29897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.184415 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "658fdc05698847c0b0f9a05d350d243b"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "4c5d9c394e35789edac3472e02ff8604"
server_key_iv: "10269bb66eecd44e5a917b17f035a923"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.184880 29897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "658fdc05698847c0b0f9a05d350d243b"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "4c5d9c394e35789edac3472e02ff8604"
server_key_iv: "10269bb66eecd44e5a917b17f035a923"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.186743 29897 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260809 02:18:33.187857 29914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.188073 29897 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:33.188176 29897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "658fdc05698847c0b0f9a05d350d243b"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "4c5d9c394e35789edac3472e02ff8604"
server_key_iv: "10269bb66eecd44e5a917b17f035a923"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.188261 29897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.211428 29897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.211742 29897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.211864 29897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.212092 29897 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:33.212472 29897 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:33.212509 29897 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.212534 29897 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:33.212589 29897 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.219081 29897 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:37699
I20260809 02:18:33.219134 30042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:37699 every 8 connection(s)
I20260809 02:18:33.219480 29897 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:33.220886 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 29897
I20260809 02:18:33.221005 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:33.221238 9377 external_mini_cluster.cc:1428] Setting key 6677b613641f52b4f0e96d0428d5ac2e
I20260809 02:18:33.222379 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--builtin_ntp_servers=127.9.40.84:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260809 02:18:33.226176 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.220346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:53119 (local address 127.9.40.126:40003)
0809 02:18:33.220420 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.220422 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.220590 (+ 168us) server_negotiation.cc:407] Connection header received
0809 02:18:33.221377 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.221378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.221414 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.221502 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.222539 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.223154 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.224571 (+ 1417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.224812 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.224893 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.225846 (+ 953us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.225848 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.225848 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:33.225854 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.225870 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:33.225874 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.225948 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.226044 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.226092 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:33.226406 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.220238 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40003 (local address 127.9.40.65:53119)
0809 02:18:33.220445 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.220488 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.221240 (+ 752us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.221623 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.221628 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.221638 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:33.222401 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.222409 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.223264 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.223265 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.224468 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.224471 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.224913 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.224915 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.225062 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.225679 (+ 617us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:33.225679 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:33.225709 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:33.225734 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:33.225949 (+ 215us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:33.225956 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:33.225960 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.226037 (+ 77us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.226148 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":106,"threads_started":1}
I20260809 02:18:33.226883 30043 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40003
I20260809 02:18:33.226984 30043 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:33.227201 30043 heartbeater.cc:511] Master 127.9.40.126:40003 requested a full tablet report, sending...
I20260809 02:18:33.227721 29829 ts_manager.cc:194] Registered new tserver with Master: 658fdc05698847c0b0f9a05d350d243b (127.9.40.65:37699)
I20260809 02:18:33.228363 29829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:53119
W20260809 02:18:33.314548 30046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.314796 30046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.314836 30046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.316105 30046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.316176 30046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.316219 30046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:33.317704 30046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:33.317981 30046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.318338 30046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.323835 30056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.323835 30052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.323886 30053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.324157 30046 server_base.cc:1034] running on GCE node
I20260809 02:18:33.324432 30046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.324690 30046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.325888 30046 hybrid_clock.cc:648] HybridClock initialized: now 1786241913325859 us; error 44 us; skew 500 ppm
I20260809 02:18:33.327349 30046 webserver.cc:460] Webserver started at http://127.9.40.66:46005/ using document root <none> and password file <none>
I20260809 02:18:33.327632 30046 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.327708 30046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.327865 30046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.329067 30046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "8758f30e0ac84e01a9715ac31abf5a40"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "8f84465dcedbd46bec100bbc473816c2"
server_key_iv: "860fe66ccb1d86ef12b7d66fe77c3b65"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.329430 30046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "8758f30e0ac84e01a9715ac31abf5a40"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "8f84465dcedbd46bec100bbc473816c2"
server_key_iv: "860fe66ccb1d86ef12b7d66fe77c3b65"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.330823 30046 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:33.331650 30062 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.331835 30046 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:33.331925 30046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "8758f30e0ac84e01a9715ac31abf5a40"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "8f84465dcedbd46bec100bbc473816c2"
server_key_iv: "860fe66ccb1d86ef12b7d66fe77c3b65"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.332010 30046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.361915 30046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.362401 30046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.362581 30046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.362900 30046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:33.363441 30046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:33.363520 30046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.363572 30046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:33.363595 30046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.369621 30046 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:39329
I20260809 02:18:33.369728 30188 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:39329 every 8 connection(s)
I20260809 02:18:33.370069 30046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:33.375679 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.371101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:42805 (local address 127.9.40.126:40003)
0809 02:18:33.371193 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.371196 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.371362 (+ 166us) server_negotiation.cc:407] Connection header received
0809 02:18:33.371858 (+ 496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.371859 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.371870 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.371943 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.372604 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.373233 (+ 629us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.374228 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.374500 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.374567 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.375415 (+ 848us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.375416 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.375416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:33.375421 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.375437 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:33.375441 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.375500 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.375584 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.375610 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260809 02:18:33.375869 30192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.370966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40003 (local address 127.9.40.66:42805)
0809 02:18:33.371221 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.371263 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.371763 (+ 500us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.371995 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.371999 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.372006 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:33.372519 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.372523 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.373323 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.373323 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.374130 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.374132 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.374598 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.374598 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.374715 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.375273 (+ 558us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:33.375273 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:33.375301 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:33.375324 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:33.375510 (+ 186us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:33.375514 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:33.375516 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.375574 (+ 58us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.375663 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":80,"threads_started":1}
I20260809 02:18:33.376423 30189 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40003
I20260809 02:18:33.376519 30189 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:33.376664 30189 heartbeater.cc:511] Master 127.9.40.126:40003 requested a full tablet report, sending...
I20260809 02:18:33.377094 29829 ts_manager.cc:194] Registered new tserver with Master: 8758f30e0ac84e01a9715ac31abf5a40 (127.9.40.66:39329)
I20260809 02:18:33.377480 29829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:42805
I20260809 02:18:33.379287 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30046
I20260809 02:18:33.379408 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:33.379634 9377 external_mini_cluster.cc:1428] Setting key a5ae6c77e4f1fe41c63a21966d123ce8
I20260809 02:18:33.380538 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--builtin_ntp_servers=127.9.40.84:34867
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:33.464778 30193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:33.465044 30193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:33.465085 30193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:33.466307 30193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:33.466377 30193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:33.466421 30193 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:33.467865 30193 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:34867
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:33.468174 30193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:33.468513 30193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:33.474048 30199 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.474159 30201 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:33.474067 30198 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:33.474097 30193 server_base.cc:1034] running on GCE node
I20260809 02:18:33.474546 30193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:33.474804 30193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:33.475984 30193 hybrid_clock.cc:648] HybridClock initialized: now 1786241913475968 us; error 29 us; skew 500 ppm
I20260809 02:18:33.477425 30193 webserver.cc:460] Webserver started at http://127.9.40.67:45749/ using document root <none> and password file <none>
I20260809 02:18:33.477690 30193 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:33.477766 30193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:33.477923 30193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:33.479033 30193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "59c224689def43a281bbd912e10a66ca"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "19ba64208e85f1570036284c78033f70"
server_key_iv: "2d1c72102d0cd5b3caf7bb7e2fe85f77"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.479417 30193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "59c224689def43a281bbd912e10a66ca"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "19ba64208e85f1570036284c78033f70"
server_key_iv: "2d1c72102d0cd5b3caf7bb7e2fe85f77"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.480906 30193 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260809 02:18:33.481765 30208 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.481969 30193 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:33.482085 30193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "59c224689def43a281bbd912e10a66ca"
format_stamp: "Formatted at 2026-08-09 02:18:33 on dist-test-slave-qg60"
server_key: "19ba64208e85f1570036284c78033f70"
server_key_iv: "2d1c72102d0cd5b3caf7bb7e2fe85f77"
server_key_version: "encryptionkey@0"
I20260809 02:18:33.482167 30193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:33.494128 30193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:33.494477 30193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:33.494648 30193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:33.494920 30193 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:33.495293 30193 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:33.495353 30193 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.495390 30193 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:33.495417 30193 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:33.501955 30193 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:37159
I20260809 02:18:33.502005 30337 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:37159 every 8 connection(s)
I20260809 02:18:33.502435 30193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:33.506574 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30193
I20260809 02:18:33.506690 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:33.506922 9377 external_mini_cluster.cc:1428] Setting key 33904e0aa4afdb7d2a1c02665229155a
I20260809 02:18:33.508149 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.503264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58001 (local address 127.9.40.126:40003)
0809 02:18:33.503344 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.503346 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.503586 (+ 240us) server_negotiation.cc:407] Connection header received
0809 02:18:33.504092 (+ 506us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.504093 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.504104 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.504167 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.504910 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.505426 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.506478 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.506795 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.506880 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.507836 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.507838 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.507838 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:33.507843 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.507858 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:33.507862 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.507931 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.508033 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.508066 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260809 02:18:33.508375 30341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.503227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40003 (local address 127.9.40.67:58001)
0809 02:18:33.503435 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.503480 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.503984 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.504220 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.504223 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.504231 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:33.504809 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.504815 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.505516 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.505516 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.506367 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.506370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.506887 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.506888 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.507031 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.507672 (+ 641us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:33.507673 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:33.507701 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:33.507726 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:33.507930 (+ 204us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:33.507937 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:33.507940 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.508024 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.508140 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":104,"threads_started":1}
I20260809 02:18:33.508893 30338 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40003
I20260809 02:18:33.508999 30338 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:33.509161 30338 heartbeater.cc:511] Master 127.9.40.126:40003 requested a full tablet report, sending...
I20260809 02:18:33.509555 29829 ts_manager.cc:194] Registered new tserver with Master: 59c224689def43a281bbd912e10a66ca (127.9.40.67:37159)
I20260809 02:18:33.509928 29829 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:58001
I20260809 02:18:33.519105 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:33.524017 29875 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.520800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53272 (local address 127.9.40.126:40003)
0809 02:18:33.520874 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.520876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.521038 (+ 162us) server_negotiation.cc:407] Connection header received
0809 02:18:33.521125 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.521126 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.521140 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.521190 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:33.521379 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.521877 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.522664 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.522881 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.522959 (+ 78us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.523707 (+ 748us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:33.523708 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:33.523709 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:33.523714 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:33.523729 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:33.523734 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:33.523849 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.523916 (+ 67us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.523927 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:18:33.525835 29829 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:53272:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:33.527853 29829 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:33.534735 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530920 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40142 (local address 127.9.40.67:37159)
0809 02:18:33.531195 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.531199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.531220 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:18:33.531285 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.531288 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.531303 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.531386 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:33.531705 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532384 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.533736 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534322 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534418 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.534514 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.534610 (+ 96us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.534646 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":236,"thread_start_us":183,"threads_started":1}
I20260809 02:18:33.534723 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:37159 (local address 127.0.0.1:40142)
0809 02:18:33.530970 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.530979 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.531018 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.531457 (+ 439us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.531458 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.531460 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:33.531506 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.531508 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532474 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.532475 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.533649 (+ 1174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.533651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534421 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.534498 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.534511 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.534596 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.534648 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":94,"threads_started":1}
I20260809 02:18:33.535147 30347 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37699 (local address 127.0.0.1:57470)
0809 02:18:33.530884 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.530895 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.530946 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.531599 (+ 653us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.531600 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.531601 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:33.531629 (+ 28us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.531629 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532884 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.532884 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.533971 (+ 1087us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.533974 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534893 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534894 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.534965 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.534968 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.535031 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.535078 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":9,"thread_start_us":112,"threads_started":1}
I20260809 02:18:33.535183 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57470 (local address 127.9.40.65:37699)
0809 02:18:33.531442 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.531447 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.531457 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:33.531508 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.531510 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.531525 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.531602 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:33.531700 (+ 98us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532799 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534086 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534806 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534891 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.534977 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.535046 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.535090 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":717,"thread_start_us":104,"threads_started":1}
I20260809 02:18:33.535320 30348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39329 (local address 127.0.0.1:55306)
0809 02:18:33.530731 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.530777 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.530847 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.531328 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.531331 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.531332 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:33.531412 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.531414 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532806 (+ 1392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.532807 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.533978 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.533979 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534962 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534963 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.535060 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.535065 (+ 5us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.535097 (+ 32us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.535136 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"mutex_wait_us":4,"thread_start_us":101,"threads_started":1}
I20260809 02:18:33.535694 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.530689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55306 (local address 127.9.40.66:39329)
0809 02:18:33.531163 (+ 474us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.531169 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.531180 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:33.531222 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.531224 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.531240 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.531310 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:33.531507 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.532674 (+ 1167us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.534086 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.534872 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.535461 (+ 589us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.535530 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.535551 (+ 21us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.535613 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":433,"thread_start_us":93,"threads_started":1}
I20260809 02:18:33.536036 29965 tablet_service.cc:1467] Processing CreateTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 (DEFAULT_TABLE table=test-table [id=2d0517c5afa34fbf8c4afe823d776e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:33.536041 30261 tablet_service.cc:1467] Processing CreateTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 (DEFAULT_TABLE table=test-table [id=2d0517c5afa34fbf8c4afe823d776e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:33.536381 29965 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34ec4b937da4cf7bfdaa7f58e033cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:33.536419 30261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34ec4b937da4cf7bfdaa7f58e033cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:33.536350 30115 tablet_service.cc:1467] Processing CreateTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 (DEFAULT_TABLE table=test-table [id=2d0517c5afa34fbf8c4afe823d776e83]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:33.536664 30115 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f34ec4b937da4cf7bfdaa7f58e033cb0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:33.539662 30353 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Bootstrap starting.
I20260809 02:18:33.540012 30354 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Bootstrap starting.
I20260809 02:18:33.540632 30354 tablet_bootstrap.cc:654] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:33.540629 30353 tablet_bootstrap.cc:654] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:33.541031 30354 log.cc:824] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:33.541033 30353 log.cc:824] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:33.541391 30356 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Bootstrap starting.
I20260809 02:18:33.541756 30354 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: No bootstrap required, opened a new log
I20260809 02:18:33.541842 30354 ts_tablet_manager.cc:1397] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:33.542021 30356 tablet_bootstrap.cc:654] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:33.542371 30356 log.cc:824] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:33.543139 30356 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: No bootstrap required, opened a new log
I20260809 02:18:33.543191 30356 ts_tablet_manager.cc:1397] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:33.543977 30354 raft_consensus.cc:348] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.544139 30354 raft_consensus.cc:374] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:33.544188 30354 raft_consensus.cc:729] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59c224689def43a281bbd912e10a66ca, State: Initialized, Role: FOLLOWER
I20260809 02:18:33.544322 30354 consensus_queue.cc:260] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.544602 30354 ts_tablet_manager.cc:1428] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:33.544696 30338 heartbeater.cc:503] Master 127.9.40.126:40003 was elected leader, sending a full tablet report...
I20260809 02:18:33.545269 30353 tablet_bootstrap.cc:492] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: No bootstrap required, opened a new log
I20260809 02:18:33.545354 30353 ts_tablet_manager.cc:1397] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260809 02:18:33.545482 30356 raft_consensus.cc:348] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.545686 30356 raft_consensus.cc:374] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:33.545729 30356 raft_consensus.cc:729] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8758f30e0ac84e01a9715ac31abf5a40, State: Initialized, Role: FOLLOWER
I20260809 02:18:33.545840 30356 consensus_queue.cc:260] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.546177 30189 heartbeater.cc:503] Master 127.9.40.126:40003 was elected leader, sending a full tablet report...
I20260809 02:18:33.546113 30356 ts_tablet_manager.cc:1428] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:33.547829 30353 raft_consensus.cc:348] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.547983 30353 raft_consensus.cc:374] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:33.548030 30353 raft_consensus.cc:729] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 658fdc05698847c0b0f9a05d350d243b, State: Initialized, Role: FOLLOWER
I20260809 02:18:33.548144 30353 consensus_queue.cc:260] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.548439 30353 ts_tablet_manager.cc:1428] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:33.548532 30043 heartbeater.cc:503] Master 127.9.40.126:40003 was elected leader, sending a full tablet report...
W20260809 02:18:33.620714 30190 tablet.cc:2369] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:33.684082 30362 raft_consensus.cc:482] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:33.684232 30362 raft_consensus.cc:504] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.684643 30362 leader_election.cc:288] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8758f30e0ac84e01a9715ac31abf5a40 (127.9.40.66:39329), 59c224689def43a281bbd912e10a66ca (127.9.40.67:37159)
I20260809 02:18:33.688220 30350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.685104 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:44949 (local address 127.9.40.66:39329)
0809 02:18:33.685212 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.685214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.685224 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:33.685259 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.685260 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.685293 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.685358 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:33.685577 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.686128 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.687336 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.687798 (+ 462us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.687915 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.687928 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.688094 (+ 166us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.688143 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:18:33.688220 30045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.684926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:39329 (local address 127.9.40.65:44949)
0809 02:18:33.685016 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.685034 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.685114 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.685363 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.685364 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.685366 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:33.685469 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.685471 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.686226 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.686228 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.687221 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.687224 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.687916 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.687917 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.687998 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.688010 (+ 12us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.688085 (+ 75us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.688135 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260809 02:18:33.688318 30352 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.685238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:45865 (local address 127.9.40.67:37159)
0809 02:18:33.685293 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.685295 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.685383 (+ 88us) server_negotiation.cc:407] Connection header received
0809 02:18:33.685488 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.685489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.685508 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.685584 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:33.685761 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.686321 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.687505 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.687910 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.688004 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.688014 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.688192 (+ 178us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.688237 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:18:33.688351 30363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.685074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:37159 (local address 127.9.40.65:45865)
0809 02:18:33.685284 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.685296 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:33.685377 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:33.685589 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:33.685590 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:33.685592 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:33.685678 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.685680 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.686424 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.686426 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.687406 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:33.687409 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.688006 (+ 597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.688008 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:33.688094 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.688097 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:33.688183 (+ 86us) client_negotiation.cc:240] Negotiation successful
0809 02:18:33.688226 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":154,"threads_started":1}
I20260809 02:18:33.688832 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34ec4b937da4cf7bfdaa7f58e033cb0" candidate_uuid: "658fdc05698847c0b0f9a05d350d243b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c224689def43a281bbd912e10a66ca" is_pre_election: true
I20260809 02:18:33.688905 30138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34ec4b937da4cf7bfdaa7f58e033cb0" candidate_uuid: "658fdc05698847c0b0f9a05d350d243b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8758f30e0ac84e01a9715ac31abf5a40" is_pre_election: true
I20260809 02:18:33.689065 30138 raft_consensus.cc:2454] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658fdc05698847c0b0f9a05d350d243b in term 0.
I20260809 02:18:33.689066 30282 raft_consensus.cc:2454] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 658fdc05698847c0b0f9a05d350d243b in term 0.
I20260809 02:18:33.689344 29917 leader_election.cc:302] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59c224689def43a281bbd912e10a66ca, 658fdc05698847c0b0f9a05d350d243b; no voters:
I20260809 02:18:33.689522 30362 raft_consensus.cc:2781] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:33.689589 30362 raft_consensus.cc:482] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:33.689626 30362 raft_consensus.cc:3037] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:33.690405 30362 raft_consensus.cc:504] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.690563 30362 leader_election.cc:288] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [CANDIDATE]: Term 1 election: Requested vote from peers 8758f30e0ac84e01a9715ac31abf5a40 (127.9.40.66:39329), 59c224689def43a281bbd912e10a66ca (127.9.40.67:37159)
I20260809 02:18:33.690747 30138 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34ec4b937da4cf7bfdaa7f58e033cb0" candidate_uuid: "658fdc05698847c0b0f9a05d350d243b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8758f30e0ac84e01a9715ac31abf5a40"
I20260809 02:18:33.690798 30282 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f34ec4b937da4cf7bfdaa7f58e033cb0" candidate_uuid: "658fdc05698847c0b0f9a05d350d243b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "59c224689def43a281bbd912e10a66ca"
I20260809 02:18:33.690855 30138 raft_consensus.cc:3037] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:33.690910 30282 raft_consensus.cc:3037] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:33.691691 30138 raft_consensus.cc:2454] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658fdc05698847c0b0f9a05d350d243b in term 1.
I20260809 02:18:33.691691 30282 raft_consensus.cc:2454] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 658fdc05698847c0b0f9a05d350d243b in term 1.
I20260809 02:18:33.691907 29917 leader_election.cc:302] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59c224689def43a281bbd912e10a66ca, 658fdc05698847c0b0f9a05d350d243b; no voters:
I20260809 02:18:33.692101 30362 raft_consensus.cc:2781] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:33.692308 30362 raft_consensus.cc:686] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [term 1 LEADER]: Becoming Leader. State: Replica: 658fdc05698847c0b0f9a05d350d243b, State: Running, Role: LEADER
I20260809 02:18:33.692425 30362 consensus_queue.cc:237] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } }
I20260809 02:18:33.693501 29829 catalog_manager.cc:5495] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b reported cstate change: term changed from 0 to 1, leader changed from <none> to 658fdc05698847c0b0f9a05d350d243b (127.9.40.65). New cstate: current_term: 1 leader_uuid: "658fdc05698847c0b0f9a05d350d243b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "658fdc05698847c0b0f9a05d350d243b" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 37699 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:33.718672 30351 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.716100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57472 (local address 127.9.40.65:37699)
0809 02:18:33.716188 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:33.716190 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:33.716201 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:33.716251 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:33.716252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:33.716274 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:33.716357 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:33.716511 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.717054 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.717868 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:33.718062 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:33.718155 (+ 93us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:33.718316 (+ 161us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:33.718404 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:33.718514 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:33.718545 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 02:18:33.718557 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 02:18:33.720209 30044 tablet.cc:2369] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:33.720930 30282 raft_consensus.cc:1261] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 1 FOLLOWER]: Refusing update from remote peer 658fdc05698847c0b0f9a05d350d243b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:33.720965 30138 raft_consensus.cc:1261] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 1 FOLLOWER]: Refusing update from remote peer 658fdc05698847c0b0f9a05d350d243b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:33.721320 30362 consensus_queue.cc:1035] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8758f30e0ac84e01a9715ac31abf5a40" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 39329 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:33.721491 30364 consensus_queue.cc:1035] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 658fdc05698847c0b0f9a05d350d243b [LEADER]: Connected to new peer: Peer: permanent_uuid: "59c224689def43a281bbd912e10a66ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 37159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:33.724063 30367 mvcc.cc:205] Tried to move back new op lower bound from 7316446878597771264 to 7316446878484901888. Current Snapshot: MvccSnapshot[applied={T|T < 7316446878597771264}]
I20260809 02:18:33.727209 29828 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:53272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:33.727306 29828 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:53272:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:33.728132 29828 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 366843ffad1b4a029b6789e2b7a2a8a3: Sending DeleteTablet for 3 replicas of tablet f34ec4b937da4cf7bfdaa7f58e033cb0
I20260809 02:18:33.728478 29965 tablet_service.cc:1514] Processing DeleteTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:33 UTC) from {username='slave'} at 127.0.0.1:57470
I20260809 02:18:33.728688 30115 tablet_service.cc:1514] Processing DeleteTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:33 UTC) from {username='slave'} at 127.0.0.1:55306
I20260809 02:18:33.728811 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29897
I20260809 02:18:33.733366 30261 tablet_service.cc:1514] Processing DeleteTablet for tablet f34ec4b937da4cf7bfdaa7f58e033cb0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:33 UTC) from {username='slave'} at 127.0.0.1:40142
I20260809 02:18:33.733724 30378 tablet_replica.cc:330] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: stopping tablet replica
I20260809 02:18:33.733866 30378 raft_consensus.cc:2229] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:33.734066 30378 raft_consensus.cc:2258] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:33.734561 30377 tablet_replica.cc:330] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: stopping tablet replica
I20260809 02:18:33.734773 30377 raft_consensus.cc:2229] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:33.734750 30378 ts_tablet_manager.cc:1905] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:33.734925 30377 raft_consensus.cc:2258] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:33.735607 30377 ts_tablet_manager.cc:1905] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:33.737762 30377 ts_tablet_manager.cc:1918] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:33.737865 30377 log.cc:1196] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/f34ec4b937da4cf7bfdaa7f58e033cb0
I20260809 02:18:33.738132 30377 ts_tablet_manager.cc:1939] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 8758f30e0ac84e01a9715ac31abf5a40: Deleting consensus metadata
I20260809 02:18:33.738709 29811 catalog_manager.cc:4841] TS 8758f30e0ac84e01a9715ac31abf5a40 (127.9.40.66:39329): tablet f34ec4b937da4cf7bfdaa7f58e033cb0 (table test-table [id=2d0517c5afa34fbf8c4afe823d776e83]) successfully deleted
W20260809 02:18:33.741557 29812 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:37699 (error 108)
I20260809 02:18:33.741796 30378 ts_tablet_manager.cc:1918] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:33.741896 30378 log.cc:1196] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/f34ec4b937da4cf7bfdaa7f58e033cb0
I20260809 02:18:33.742147 30349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:33.741972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:37699 (local address 127.0.0.1:57478)
0809 02:18:33.742051 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:33.742108 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37699: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260809 02:18:33.742216 30378 ts_tablet_manager.cc:1939] T f34ec4b937da4cf7bfdaa7f58e033cb0 P 59c224689def43a281bbd912e10a66ca: Deleting consensus metadata
W20260809 02:18:33.742300 29812 catalog_manager.cc:4584] TS 658fdc05698847c0b0f9a05d350d243b (127.9.40.65:37699): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f34ec4b937da4cf7bfdaa7f58e033cb0: Network error: Client connection negotiation failed: client connection to 127.9.40.65:37699: connect: Connection refused (error 111)
I20260809 02:18:33.742678 29811 catalog_manager.cc:4841] TS 59c224689def43a281bbd912e10a66ca (127.9.40.67:37159): tablet f34ec4b937da4cf7bfdaa7f58e033cb0 (table test-table [id=2d0517c5afa34fbf8c4afe823d776e83]) successfully deleted
I20260809 02:18:33.742738 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30046
I20260809 02:18:33.753162 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30193
I20260809 02:18:33.761027 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 29793
2026-08-09T02:18:33Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (867 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:33 dist-test-slave-qg60 krb5kdc[30382](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:33 dist-test-slave-qg60 krb5kdc[30382](info): set up 2 sockets
Aug 09 02:18:33 dist-test-slave-qg60 krb5kdc[30382](info): commencing operation
krb5kdc: starting...
W20260809 02:18:34.873113 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:34 dist-test-slave-qg60 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:35.009164 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40477
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:40477
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:35.098694 30396 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.098903 30396 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.098927 30396 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.100121 30396 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:35.100167 30396 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.100180 30396 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.100193 30396 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:35.100205 30396 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:35.101861 30396 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:40477
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:40477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30396
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:35.102136 30396 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.102375 30396 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:35.107628 30396 server_base.cc:1034] running on GCE node
W20260809 02:18:35.107576 30401 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.107524 30402 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.107550 30405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.108055 30396 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.108353 30396 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.109529 30396 hybrid_clock.cc:648] HybridClock initialized: now 1786241915109519 us; error 60 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.112221 30396 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:35.112811 30396 webserver.cc:460] Webserver started at http://127.9.40.126:36765/ using document root <none> and password file <none>
I20260809 02:18:35.113070 30396 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.113142 30396 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.113310 30396 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:35.114399 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "8bacf53d8c8041d7882a1c3d7f171237"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "15f3b0c5330f29e3a51723f862eef2c4"
server_key_iv: "5b913af0aae8e3c0dec815960bbbcad4"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.114786 30396 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "8bacf53d8c8041d7882a1c3d7f171237"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "15f3b0c5330f29e3a51723f862eef2c4"
server_key_iv: "5b913af0aae8e3c0dec815960bbbcad4"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.116243 30396 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:18:35.117156 30412 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.117342 30396 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260809 02:18:35.117434 30396 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "8bacf53d8c8041d7882a1c3d7f171237"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "15f3b0c5330f29e3a51723f862eef2c4"
server_key_iv: "5b913af0aae8e3c0dec815960bbbcad4"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.117568 30396 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.161657 30396 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.164076 30396 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.164248 30396 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.168721 30396 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:40477
I20260809 02:18:35.168789 30472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:40477 every 8 connection(s)
I20260809 02:18:35.169174 30396 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:35.170084 30473 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.172395 30473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: Bootstrap starting.
I20260809 02:18:35.172981 30473 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.173218 30473 log.cc:824] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.173905 30473 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: No bootstrap required, opened a new log
I20260809 02:18:35.175458 30473 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } }
I20260809 02:18:35.175603 30473 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.175626 30473 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8bacf53d8c8041d7882a1c3d7f171237, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.175765 30473 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } }
I20260809 02:18:35.175846 30473 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:35.175869 30473 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:35.175896 30473 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.176139 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30396
I20260809 02:18:35.176258 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:35.176473 9377 external_mini_cluster.cc:1428] Setting key 3fd99aef192503c98f3d09d248c4d8ee
I20260809 02:18:35.176659 30473 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } }
I20260809 02:18:35.176815 30473 leader_election.cc:302] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8bacf53d8c8041d7882a1c3d7f171237; no voters:
I20260809 02:18:35.177006 30473 leader_election.cc:288] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:35.177050 30477 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:35.177181 30477 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [term 1 LEADER]: Becoming Leader. State: Replica: 8bacf53d8c8041d7882a1c3d7f171237, State: Running, Role: LEADER
I20260809 02:18:35.177273 30473 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:35.177250 30477 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } }
I20260809 02:18:35.177800 30480 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8bacf53d8c8041d7882a1c3d7f171237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } } }
I20260809 02:18:35.177906 30480 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:35.178069 30478 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8bacf53d8c8041d7882a1c3d7f171237. Latest consensus state: current_term: 1 leader_uuid: "8bacf53d8c8041d7882a1c3d7f171237" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8bacf53d8c8041d7882a1c3d7f171237" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 40477 } } }
I20260809 02:18:35.178143 30478 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:35.178514 30488 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:35.179145 30488 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:35.180852 30488 catalog_manager.cc:1347] Generated new cluster ID: 88d7fee8e5a643bc98fe8ba501dce0ba
I20260809 02:18:35.180907 30488 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241914, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:35.190295 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.177087 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54442 (local address 127.9.40.126:40477)
0809 02:18:35.177276 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.177286 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.177639 (+ 353us) server_negotiation.cc:407] Connection header received
0809 02:18:35.177763 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.177771 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.177828 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.178084 (+ 256us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.179176 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.180582 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.181697 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.182305 (+ 608us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.182413 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.185397 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.185402 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.185402 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.185415 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.188437 (+ 3022us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.188877 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.188881 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.188881 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.189033 (+ 152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.189251 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.189252 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.189252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.189616 (+ 364us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.189799 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.189883 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.190053 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130,"thread_start_us":96,"threads_started":1}
I20260809 02:18:35.198619 30488 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:35.199339 30488 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:35.205428 30488 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: Generated new TSK 0
I20260809 02:18:35.205631 30488 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.9.40.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
Entry for principal kudu/127.9.40.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab.
I20260809 02:18:35.240471 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:35.324651 30501 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.324936 30501 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.324966 30501 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.326126 30501 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.326164 30501 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.326191 30501 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:35.327534 30501 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.65.keytab
--principal=kudu/127.9.40.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30501
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:35.327710 30501 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.327883 30501 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.333037 30506 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.333037 30507 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.333081 30509 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.333348 30501 server_base.cc:1034] running on GCE node
I20260809 02:18:35.333551 30501 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.333828 30501 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.335031 30501 hybrid_clock.cc:648] HybridClock initialized: now 1786241915335011 us; error 31 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.337769 30501 init.cc:377] Logged in from keytab as kudu/127.9.40.65@KRBTEST.COM (short username kudu)
I20260809 02:18:35.338325 30501 webserver.cc:460] Webserver started at http://127.9.40.65:36175/ using document root <none> and password file <none>
I20260809 02:18:35.338564 30501 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.338636 30501 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.338799 30501 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:35.339908 30501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "183ffc2505cb4240b2164c593d0cde26"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "e27f3a5191583a5c897bfc20b5166709"
server_key_iv: "25a3a2a05c01a3a1ff267414314efe5a"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.340310 30501 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "183ffc2505cb4240b2164c593d0cde26"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "e27f3a5191583a5c897bfc20b5166709"
server_key_iv: "25a3a2a05c01a3a1ff267414314efe5a"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.341871 30501 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:35.342701 30518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.342924 30501 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:35.343079 30501 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "183ffc2505cb4240b2164c593d0cde26"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "e27f3a5191583a5c897bfc20b5166709"
server_key_iv: "25a3a2a05c01a3a1ff267414314efe5a"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.343170 30501 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.369988 30501 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.371986 30501 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.372176 30501 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.372448 30501 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.372841 30501 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:35.372901 30501 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.372941 30501 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:35.372968 30501 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.379510 30501 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:38689
I20260809 02:18:35.379599 30645 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:38689 every 8 connection(s)
I20260809 02:18:35.379940 30501 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.65@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:35.386685 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30501
I20260809 02:18:35.386816 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:35.387059 9377 external_mini_cluster.cc:1428] Setting key c855107bbb721076a351d60a9f3c4d23
I20260809 02:18:35.390394 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.380790 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:46787 (local address 127.9.40.126:40477)
0809 02:18:35.380856 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.380858 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.381129 (+ 271us) server_negotiation.cc:407] Connection header received
0809 02:18:35.381568 (+ 439us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.381569 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.381583 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.381648 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.382708 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.383256 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.384259 (+ 1003us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.384450 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.384526 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.386919 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.386921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.386921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.386931 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.388549 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.389174 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.389175 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.389175 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.389235 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.389557 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.389557 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.389558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.389700 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.389777 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.390154 (+ 377us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.390269 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:18:35.390419 30648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.380738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40477 (local address 127.9.40.65:46787)
0809 02:18:35.380930 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.380975 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.381473 (+ 498us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.381709 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.381712 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.382380 (+ 668us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.382597 (+ 217us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.382602 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.383352 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.383353 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.384173 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.384176 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.384524 (+ 348us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.384525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.384639 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.385167 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.385175 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.386754 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.388646 (+ 1892us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.388656 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.388659 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.389079 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.389311 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.389312 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.389312 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.389473 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.389782 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.389790 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.389986 (+ 196us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.390082 (+ 96us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.390235 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":93,"threads_started":1}
I20260809 02:18:35.391016 30646 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40477
I20260809 02:18:35.391158 30646 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.391371 30646 heartbeater.cc:511] Master 127.9.40.126:40477 requested a full tablet report, sending...
I20260809 02:18:35.391952 30433 ts_manager.cc:194] Registered new tserver with Master: 183ffc2505cb4240b2164c593d0cde26 (127.9.40.65:38689)
I20260809 02:18:35.392654 30433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.65@KRBTEST.COM'} at 127.9.40.65:46787
WARNING: no policy specified for kudu/127.9.40.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
Entry for principal kudu/127.9.40.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab.
I20260809 02:18:35.416049 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:35.498878 30652 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.499135 30652 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.499177 30652 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.500346 30652 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.500419 30652 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.500463 30652 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:35.502030 30652 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.66.keytab
--principal=kudu/127.9.40.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30652
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:35.502327 30652 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.502596 30652 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.507980 30658 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.508049 30660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.508004 30657 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.508381 30652 server_base.cc:1034] running on GCE node
I20260809 02:18:35.508589 30652 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.508857 30652 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.510001 30652 hybrid_clock.cc:648] HybridClock initialized: now 1786241915509990 us; error 27 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.513176 30652 init.cc:377] Logged in from keytab as kudu/127.9.40.66@KRBTEST.COM (short username kudu)
I20260809 02:18:35.513726 30652 webserver.cc:460] Webserver started at http://127.9.40.66:44701/ using document root <none> and password file <none>
I20260809 02:18:35.513991 30652 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.514039 30652 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.514178 30652 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:35.515264 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "2ed8e8af615249fa9cfd6fe5b75f2087"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "7167130ed4a468210f9d394855321c4d"
server_key_iv: "97d6595b4d36b0c7f3128d70a3edb4ca"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.515642 30652 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "2ed8e8af615249fa9cfd6fe5b75f2087"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "7167130ed4a468210f9d394855321c4d"
server_key_iv: "97d6595b4d36b0c7f3128d70a3edb4ca"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.517084 30652 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:18:35.517884 30667 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.518072 30652 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:18:35.518182 30652 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "2ed8e8af615249fa9cfd6fe5b75f2087"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "7167130ed4a468210f9d394855321c4d"
server_key_iv: "97d6595b4d36b0c7f3128d70a3edb4ca"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.518260 30652 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.532335 30652 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.534718 30652 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.534891 30652 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.535136 30652 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.535446 30652 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:35.535507 30652 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.535548 30652 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:35.535576 30652 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.541724 30652 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:35173
I20260809 02:18:35.541770 30790 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:35173 every 8 connection(s)
I20260809 02:18:35.542143 30652 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.66@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:35.551819 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.542998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:47627 (local address 127.9.40.126:40477)
0809 02:18:35.543068 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.543070 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.543218 (+ 148us) server_negotiation.cc:407] Connection header received
0809 02:18:35.543730 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.543731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.543742 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.543813 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.544782 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.545492 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.546608 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.546798 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.546864 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.549132 (+ 2268us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.549133 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.549133 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.549140 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.550393 (+ 1253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.550870 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.550871 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.550871 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.550941 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.551168 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.551169 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.551169 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.551282 (+ 113us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.551342 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.551643 (+ 301us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.551752 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:35.552003 30793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.542912 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40477 (local address 127.9.40.66:47627)
0809 02:18:35.543087 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.543132 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.543611 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.543866 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.543869 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.544425 (+ 556us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.544629 (+ 204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.544633 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.545598 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.545599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.546517 (+ 918us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.546519 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.546880 (+ 361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.546881 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.546976 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.547592 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.547600 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.549045 (+ 1445us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.550471 (+ 1426us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.550477 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.550502 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.550717 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.551013 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.551013 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.551013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.551109 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.551347 (+ 238us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.551352 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.551467 (+ 115us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.551582 (+ 115us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.551710 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":108,"threads_started":1}
I20260809 02:18:35.551970 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30652
I20260809 02:18:35.552152 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:35.552384 9377 external_mini_cluster.cc:1428] Setting key 5b4d3924fe8e420b25b713627f183667
I20260809 02:18:35.552841 30791 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40477
I20260809 02:18:35.552963 30791 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.553200 30791 heartbeater.cc:511] Master 127.9.40.126:40477 requested a full tablet report, sending...
I20260809 02:18:35.553588 30433 ts_manager.cc:194] Registered new tserver with Master: 2ed8e8af615249fa9cfd6fe5b75f2087 (127.9.40.66:35173)
I20260809 02:18:35.553970 30433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.66@KRBTEST.COM'} at 127.9.40.66:47627
WARNING: no policy specified for kudu/127.9.40.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
Entry for principal kudu/127.9.40.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab.
I20260809 02:18:35.583531 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:43639
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:35.668080 30797 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:35.668300 30797 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:35.668324 30797 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:35.669579 30797 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:35.669627 30797 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:35.669654 30797 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:35.671068 30797 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:43639
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.67.keytab
--principal=kudu/127.9.40.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:40477
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30797
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:35.671257 30797 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:35.671437 30797 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:35.676487 30807 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.676497 30803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:35.676573 30804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:35.676764 30797 server_base.cc:1034] running on GCE node
I20260809 02:18:35.677131 30797 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:35.677389 30797 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:35.678603 30797 hybrid_clock.cc:648] HybridClock initialized: now 1786241915678579 us; error 37 us; skew 500 ppm
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:35.681313 30797 init.cc:377] Logged in from keytab as kudu/127.9.40.67@KRBTEST.COM (short username kudu)
I20260809 02:18:35.681808 30797 webserver.cc:460] Webserver started at http://127.9.40.67:38907/ using document root <none> and password file <none>
I20260809 02:18:35.682039 30797 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:35.682111 30797 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:35.682268 30797 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:35.683350 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "fef9d826c7b74160847236efb1dec0d4"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "0751323877bfc2420b15702d17586f68"
server_key_iv: "6a8bcc02e9620134cd319ba3ebafbd83"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.683720 30797 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "fef9d826c7b74160847236efb1dec0d4"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "0751323877bfc2420b15702d17586f68"
server_key_iv: "6a8bcc02e9620134cd319ba3ebafbd83"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.685163 30797 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:18:35.685932 30816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.686110 30797 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:18:35.686216 30797 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "fef9d826c7b74160847236efb1dec0d4"
format_stamp: "Formatted at 2026-08-09 02:18:35 on dist-test-slave-qg60"
server_key: "0751323877bfc2420b15702d17586f68"
server_key_iv: "6a8bcc02e9620134cd319ba3ebafbd83"
server_key_version: "encryptionkey@0"
I20260809 02:18:35.686290 30797 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:35.712109 30797 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:35.715677 30797 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:35.715860 30797 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:35.716152 30797 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:35.716488 30797 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:35.716521 30797 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.716550 30797 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:35.716564 30797 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:35.723011 30797 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:43909
I20260809 02:18:35.723065 30944 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:43909 every 8 connection(s)
I20260809 02:18:35.723388 30797 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:35.729694 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 30797
I20260809 02:18:35.729779 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
Aug 09 02:18:35 dist-test-slave-qg60 krb5kdc[30382](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241915, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.67@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:35.729977 9377 external_mini_cluster.cc:1428] Setting key 2d7b18125d95e868213f5a073d724542
I20260809 02:18:35.733214 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.724127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41379 (local address 127.9.40.126:40477)
0809 02:18:35.724177 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.724179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.724413 (+ 234us) server_negotiation.cc:407] Connection header received
0809 02:18:35.725030 (+ 617us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.725032 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.725044 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.725105 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.726082 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.726587 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.727601 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.727792 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.727898 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.730335 (+ 2437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.730336 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.730337 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.730345 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.731768 (+ 1423us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.732188 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.732189 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.732189 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.732254 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.732514 (+ 260us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.732515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.732515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.732644 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.732716 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.733033 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.733115 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:18:35.733430 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.724103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:40477 (local address 127.9.40.67:41379)
0809 02:18:35.724287 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.724325 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.724921 (+ 596us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.725184 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.725188 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.725765 (+ 577us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:35.725959 (+ 194us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.725963 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.726686 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.726687 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.727504 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.727506 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.727907 (+ 401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.727908 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.728031 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.728542 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:18:35.728552 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:35.730228 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:35.731860 (+ 1632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.731868 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.731870 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.732107 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.732326 (+ 219us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:18:35.732327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:18:35.732327 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:18:35.732437 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:18:35.732719 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:35.732743 (+ 24us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:35.732885 (+ 142us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.732998 (+ 113us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.733178 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":87,"threads_started":1}
I20260809 02:18:35.733963 30945 heartbeater.cc:347] Connected to a master server at 127.9.40.126:40477
I20260809 02:18:35.734069 30945 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:35.734256 30945 heartbeater.cc:511] Master 127.9.40.126:40477 requested a full tablet report, sending...
I20260809 02:18:35.734719 30433 ts_manager.cc:194] Registered new tserver with Master: fef9d826c7b74160847236efb1dec0d4 (127.9.40.67:43909)
I20260809 02:18:35.735072 30433 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.9.40.67@KRBTEST.COM'} at 127.9.40.67:41379
I20260809 02:18:35.741843 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:35.749805 30481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.743384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54450 (local address 127.9.40.126:40477)
0809 02:18:35.743469 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.743471 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.743497 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:18:35.743588 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.743589 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.743598 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.743641 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:35.744223 (+ 582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.744711 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.745654 (+ 943us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.745851 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.745906 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.747319 (+ 1413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:35.747320 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:35.747320 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:35.747326 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:35.748627 (+ 1301us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.749037 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.749038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.749038 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.749095 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:35.749279 (+ 184us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:35.749280 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:35.749280 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:35.749423 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:35.749496 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.749622 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.749683 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260809 02:18:35.751619 30433 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54450:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:35.753291 30433 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:35.760409 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.756049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47110 (local address 127.9.40.67:43909)
0809 02:18:35.756291 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.756295 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.756304 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:18:35.756365 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.756367 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.756383 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.756459 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.756663 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.757803 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.759495 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.759983 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.760090 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.760156 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.760285 (+ 129us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.760323 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":122,"threads_started":1}
I20260809 02:18:35.760398 30961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.755782 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:43909 (local address 127.0.0.1:47110)
0809 02:18:35.756192 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.756201 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.756240 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.756538 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.756538 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.756540 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.756581 (+ 41us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.756583 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.757937 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.757938 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.759415 (+ 1477us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.759417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.760095 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.760096 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.760178 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.760191 (+ 13us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.760265 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.760321 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":31,"thread_start_us":57,"threads_started":1}
I20260809 02:18:35.760581 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.755990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.9.40.65:38689)
0809 02:18:35.756204 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.756210 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.756222 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:18:35.756271 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.756273 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.756289 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.756382 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.756581 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.757846 (+ 1265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.759402 (+ 1556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.760134 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.760202 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.760251 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.760446 (+ 195us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.760485 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":138,"threads_started":1}
I20260809 02:18:35.760841 30959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.755777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38689 (local address 127.0.0.1:37662)
0809 02:18:35.755957 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.756011 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.756080 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.756395 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.756398 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.756401 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.756477 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.756479 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.757940 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.757940 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.759314 (+ 1374us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.759317 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.760285 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.760286 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.760347 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.760351 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.760694 (+ 343us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.760776 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"mutex_wait_us":49,"thread_start_us":99,"threads_started":1}
I20260809 02:18:35.761552 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.755777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35173 (local address 127.0.0.1:51866)
0809 02:18:35.756441 (+ 664us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.756452 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.756521 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.758168 (+ 1647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.758169 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.758171 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.758215 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.758217 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.759246 (+ 1029us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.759247 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.760365 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.760367 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.761299 (+ 932us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.761300 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.761360 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.761364 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.761438 (+ 74us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.761475 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"mutex_wait_us":19,"thread_start_us":82,"threads_started":1}
I20260809 02:18:35.761564 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.756257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51866 (local address 127.9.40.66:35173)
0809 02:18:35.758018 (+ 1761us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.758022 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.758030 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:18:35.758060 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.758062 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.758075 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.758166 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.758286 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.759145 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.760454 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.761238 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.761292 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.761326 (+ 34us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.761464 (+ 138us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.761494 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1730,"thread_start_us":93,"threads_started":1}
I20260809 02:18:35.761582 30869 tablet_service.cc:1467] Processing CreateTablet for tablet f95bb37090a54d36b161a2c2302c25b9 (DEFAULT_TABLE table=test-table [id=4413851a48384ccdb351fa37fe6bed12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.761989 30869 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95bb37090a54d36b161a2c2302c25b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.762255 30719 tablet_service.cc:1467] Processing CreateTablet for tablet f95bb37090a54d36b161a2c2302c25b9 (DEFAULT_TABLE table=test-table [id=4413851a48384ccdb351fa37fe6bed12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.762598 30719 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95bb37090a54d36b161a2c2302c25b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.765538 30966 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Bootstrap starting.
I20260809 02:18:35.766162 30569 tablet_service.cc:1467] Processing CreateTablet for tablet f95bb37090a54d36b161a2c2302c25b9 (DEFAULT_TABLE table=test-table [id=4413851a48384ccdb351fa37fe6bed12]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:35.766270 30966 tablet_bootstrap.cc:654] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.766460 30569 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f95bb37090a54d36b161a2c2302c25b9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:35.766912 30966 log.cc:824] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.767717 30966 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: No bootstrap required, opened a new log
I20260809 02:18:35.767797 30966 ts_tablet_manager.cc:1397] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.769114 30968 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Bootstrap starting.
I20260809 02:18:35.769726 30968 tablet_bootstrap.cc:654] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.769750 30971 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Bootstrap starting.
I20260809 02:18:35.769935 30966 raft_consensus.cc:348] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.770123 30966 raft_consensus.cc:374] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.770170 30966 raft_consensus.cc:729] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fef9d826c7b74160847236efb1dec0d4, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.770321 30966 consensus_queue.cc:260] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.770367 30968 log.cc:824] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.770555 30971 tablet_bootstrap.cc:654] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:35.770612 30966 ts_tablet_manager.cc:1428] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:35.770751 30945 heartbeater.cc:503] Master 127.9.40.126:40477 was elected leader, sending a full tablet report...
I20260809 02:18:35.771136 30971 log.cc:824] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:35.771358 30968 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: No bootstrap required, opened a new log
I20260809 02:18:35.771428 30968 ts_tablet_manager.cc:1397] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.772032 30971 tablet_bootstrap.cc:492] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: No bootstrap required, opened a new log
I20260809 02:18:35.772102 30971 ts_tablet_manager.cc:1397] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260809 02:18:35.773473 30968 raft_consensus.cc:348] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.773649 30968 raft_consensus.cc:374] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.773689 30968 raft_consensus.cc:729] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ed8e8af615249fa9cfd6fe5b75f2087, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.773800 30968 consensus_queue.cc:260] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.774088 30968 ts_tablet_manager.cc:1428] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:35.774122 30791 heartbeater.cc:503] Master 127.9.40.126:40477 was elected leader, sending a full tablet report...
I20260809 02:18:35.774204 30971 raft_consensus.cc:348] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.774371 30971 raft_consensus.cc:374] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:35.774415 30971 raft_consensus.cc:729] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 183ffc2505cb4240b2164c593d0cde26, State: Initialized, Role: FOLLOWER
I20260809 02:18:35.774520 30971 consensus_queue.cc:260] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.774801 30971 ts_tablet_manager.cc:1428] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:35.774814 30646 heartbeater.cc:503] Master 127.9.40.126:40477 was elected leader, sending a full tablet report...
W20260809 02:18:35.792764 30792 tablet.cc:2369] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:35.853909 30972 raft_consensus.cc:482] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:35.854056 30972 raft_consensus.cc:504] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.854566 30972 leader_election.cc:288] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 183ffc2505cb4240b2164c593d0cde26 (127.9.40.65:38689), 2ed8e8af615249fa9cfd6fe5b75f2087 (127.9.40.66:35173)
I20260809 02:18:35.857424 30963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.854809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:58157 (local address 127.9.40.65:38689)
0809 02:18:35.854871 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.854873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.854883 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:35.854913 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.854913 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.854927 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.855029 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.855199 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.855682 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.856656 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.857052 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.857140 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.857152 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.857310 (+ 158us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.857351 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:18:35.857463 30948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.854707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38689 (local address 127.9.40.67:58157)
0809 02:18:35.854764 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.854777 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.854823 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.855048 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.855049 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.855051 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.855124 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.855125 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.855774 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.855775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.856553 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.856556 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.857139 (+ 583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.857140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.857220 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.857223 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.857299 (+ 76us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.857346 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:18:35.857475 30965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.854817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:41479 (local address 127.9.40.66:35173)
0809 02:18:35.854870 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.854872 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.855010 (+ 138us) server_negotiation.cc:407] Connection header received
0809 02:18:35.855082 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.855084 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.855097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.855176 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:35.855325 (+ 149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.855789 (+ 464us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.856701 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.857123 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.857207 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.857217 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.857366 (+ 149us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.857403 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:18:35.857573 30978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.854757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:35173 (local address 127.9.40.67:41479)
0809 02:18:35.854950 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.854957 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:35.855027 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:35.855182 (+ 155us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:35.855182 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:35.855184 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:35.855257 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.855258 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.855876 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.855878 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.856610 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:35.856612 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.857211 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.857212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:35.857290 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.857293 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:35.857360 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:18:35.857400 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":128,"threads_started":1}
I20260809 02:18:35.857913 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95bb37090a54d36b161a2c2302c25b9" candidate_uuid: "fef9d826c7b74160847236efb1dec0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183ffc2505cb4240b2164c593d0cde26" is_pre_election: true
I20260809 02:18:35.857935 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95bb37090a54d36b161a2c2302c25b9" candidate_uuid: "fef9d826c7b74160847236efb1dec0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" is_pre_election: true
I20260809 02:18:35.858100 30742 raft_consensus.cc:2454] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef9d826c7b74160847236efb1dec0d4 in term 0.
I20260809 02:18:35.858109 30590 raft_consensus.cc:2454] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fef9d826c7b74160847236efb1dec0d4 in term 0.
I20260809 02:18:35.858357 30821 leader_election.cc:302] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ed8e8af615249fa9cfd6fe5b75f2087, fef9d826c7b74160847236efb1dec0d4; no voters:
I20260809 02:18:35.858508 30972 raft_consensus.cc:2781] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:35.858608 30972 raft_consensus.cc:482] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:35.858624 30972 raft_consensus.cc:3037] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.859382 30972 raft_consensus.cc:504] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.859544 30972 leader_election.cc:288] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [CANDIDATE]: Term 1 election: Requested vote from peers 183ffc2505cb4240b2164c593d0cde26 (127.9.40.65:38689), 2ed8e8af615249fa9cfd6fe5b75f2087 (127.9.40.66:35173)
I20260809 02:18:35.859730 30590 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95bb37090a54d36b161a2c2302c25b9" candidate_uuid: "fef9d826c7b74160847236efb1dec0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "183ffc2505cb4240b2164c593d0cde26"
I20260809 02:18:35.859773 30742 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f95bb37090a54d36b161a2c2302c25b9" candidate_uuid: "fef9d826c7b74160847236efb1dec0d4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087"
I20260809 02:18:35.859810 30590 raft_consensus.cc:3037] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.859867 30742 raft_consensus.cc:3037] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:35.860479 30590 raft_consensus.cc:2454] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef9d826c7b74160847236efb1dec0d4 in term 1.
I20260809 02:18:35.860682 30819 leader_election.cc:302] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 183ffc2505cb4240b2164c593d0cde26, fef9d826c7b74160847236efb1dec0d4; no voters:
I20260809 02:18:35.860854 30972 raft_consensus.cc:2781] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:35.860853 30742 raft_consensus.cc:2454] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fef9d826c7b74160847236efb1dec0d4 in term 1.
I20260809 02:18:35.861045 30972 raft_consensus.cc:686] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 1 LEADER]: Becoming Leader. State: Replica: fef9d826c7b74160847236efb1dec0d4, State: Running, Role: LEADER
I20260809 02:18:35.861151 30972 consensus_queue.cc:237] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } }
I20260809 02:18:35.862052 30433 catalog_manager.cc:5495] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 reported cstate change: term changed from 0 to 1, leader changed from <none> to fef9d826c7b74160847236efb1dec0d4 (127.9.40.67). New cstate: current_term: 1 leader_uuid: "fef9d826c7b74160847236efb1dec0d4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fef9d826c7b74160847236efb1dec0d4" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 43909 } health_report { overall_health: HEALTHY } } }
I20260809 02:18:35.877562 30964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.874824 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47126 (local address 127.9.40.67:43909)
0809 02:18:35.874895 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:35.874897 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:35.874915 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:18:35.874991 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:35.874992 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:35.875016 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:35.875107 (+ 91us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:35.875258 (+ 151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.875744 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.876578 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:35.876872 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:35.876964 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:35.877151 (+ 187us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:35.877270 (+ 119us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:35.877371 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:35.877391 (+ 20us) server_negotiation.cc:299] Negotiation successful
0809 02:18:35.877415 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:35.879885 30590 raft_consensus.cc:1261] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 1 FOLLOWER]: Refusing update from remote peer fef9d826c7b74160847236efb1dec0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:35.879920 30742 raft_consensus.cc:1261] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 1 FOLLOWER]: Refusing update from remote peer fef9d826c7b74160847236efb1dec0d4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:35.880306 30972 consensus_queue.cc:1035] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "183ffc2505cb4240b2164c593d0cde26" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 38689 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:35.880492 30979 consensus_queue.cc:1035] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ed8e8af615249fa9cfd6fe5b75f2087" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 35173 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260809 02:18:35.880793 30647 tablet.cc:2369] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260809 02:18:35.883741 30982 mvcc.cc:205] Tried to move back new op lower bound from 7316446887440494592 to 7316446887367958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316446887440494592 or (T in {7316446887440494592})}]
I20260809 02:18:35.884191 30984 mvcc.cc:205] Tried to move back new op lower bound from 7316446887440494592 to 7316446887367958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316446887440494592}]
I20260809 02:18:35.883862 30983 mvcc.cc:205] Tried to move back new op lower bound from 7316446887440494592 to 7316446887367958528. Current Snapshot: MvccSnapshot[applied={T|T < 7316446887440494592}]
I20260809 02:18:35.885639 30432 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:35.885761 30432 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54450:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:35.886523 30432 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 8bacf53d8c8041d7882a1c3d7f171237: Sending DeleteTablet for 3 replicas of tablet f95bb37090a54d36b161a2c2302c25b9
I20260809 02:18:35.886861 30569 tablet_service.cc:1514] Processing DeleteTablet for tablet f95bb37090a54d36b161a2c2302c25b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:37662
I20260809 02:18:35.886996 30719 tablet_service.cc:1514] Processing DeleteTablet for tablet f95bb37090a54d36b161a2c2302c25b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:51866
I20260809 02:18:35.887086 30993 tablet_replica.cc:330] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26: stopping tablet replica
I20260809 02:18:35.887216 30869 tablet_service.cc:1514] Processing DeleteTablet for tablet f95bb37090a54d36b161a2c2302c25b9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:35 UTC) from {username='kudu', principal='kudu/127.9.40.126@KRBTEST.COM'} at 127.0.0.1:47110
I20260809 02:18:35.887277 30993 raft_consensus.cc:2229] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:35.887384 30993 raft_consensus.cc:2258] T f95bb37090a54d36b161a2c2302c25b9 P 183ffc2505cb4240b2164c593d0cde26 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.887492 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30501
I20260809 02:18:35.887555 30994 tablet_replica.cc:330] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: stopping tablet replica
I20260809 02:18:35.893950 30994 raft_consensus.cc:2229] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:35.894083 30994 raft_consensus.cc:2258] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.897478 30994 ts_tablet_manager.cc:1905] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:35.899683 30994 ts_tablet_manager.cc:1918] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:35.899778 30994 log.cc:1196] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-1/wal/wals/f95bb37090a54d36b161a2c2302c25b9
I20260809 02:18:35.900068 30994 ts_tablet_manager.cc:1939] T f95bb37090a54d36b161a2c2302c25b9 P 2ed8e8af615249fa9cfd6fe5b75f2087: Deleting consensus metadata
I20260809 02:18:35.900578 30416 catalog_manager.cc:4841] TS 2ed8e8af615249fa9cfd6fe5b75f2087 (127.9.40.66:35173): tablet f95bb37090a54d36b161a2c2302c25b9 (table test-table [id=4413851a48384ccdb351fa37fe6bed12]) successfully deleted
I20260809 02:18:35.900857 30995 tablet_replica.cc:330] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: stopping tablet replica
I20260809 02:18:35.900995 30995 raft_consensus.cc:2229] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:35.901118 30995 raft_consensus.cc:2258] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:35.901611 30995 ts_tablet_manager.cc:1905] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260809 02:18:35.903236 30415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:38689 (error 108)
I20260809 02:18:35.903505 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30652
I20260809 02:18:35.904749 30995 ts_tablet_manager.cc:1918] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:35.904815 30958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:35.903772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:38689 (local address 127.0.0.1:37664)
0809 02:18:35.904699 (+ 927us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:35.904775 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":898}
I20260809 02:18:35.904847 30995 log.cc:1196] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/f95bb37090a54d36b161a2c2302c25b9
I20260809 02:18:35.905149 30995 ts_tablet_manager.cc:1939] T f95bb37090a54d36b161a2c2302c25b9 P fef9d826c7b74160847236efb1dec0d4: Deleting consensus metadata
I20260809 02:18:35.909003 30414 catalog_manager.cc:4841] TS fef9d826c7b74160847236efb1dec0d4 (127.9.40.67:43909): tablet f95bb37090a54d36b161a2c2302c25b9 (table test-table [id=4413851a48384ccdb351fa37fe6bed12]) successfully deleted
W20260809 02:18:35.909564 30415 catalog_manager.cc:4584] TS 183ffc2505cb4240b2164c593d0cde26 (127.9.40.65:38689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f95bb37090a54d36b161a2c2302c25b9: Network error: Client connection negotiation failed: client connection to 127.9.40.65:38689: connect: Connection refused (error 111)
I20260809 02:18:35.909802 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30797
I20260809 02:18:35.918190 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 30396
2026-08-09T02:18:35Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2158 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-09T02:18:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:35Z Disabled control of system clock
I20260809 02:18:35.950389 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38865
--webserver_interface=127.9.40.126
--webserver_port=0
--builtin_ntp_servers=127.9.40.84:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38865
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:36.047739 31000 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.048064 31000 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.048101 31000 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.049767 31000 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:36.049817 31000 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.049830 31000 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.049844 31000 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:36.049856 31000 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:36.051281 31000 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38865
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31000
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:36.051563 31000 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.051882 31000 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.056828 31005 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.056838 31006 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.057022 31000 server_base.cc:1034] running on GCE node
W20260809 02:18:36.056866 31008 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.057346 31000 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.057648 31000 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.058802 31000 hybrid_clock.cc:648] HybridClock initialized: now 1786241916058787 us; error 25 us; skew 500 ppm
I20260809 02:18:36.060138 31000 webserver.cc:460] Webserver started at http://127.9.40.126:33827/ using document root <none> and password file <none>
I20260809 02:18:36.060396 31000 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.060451 31000 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.060619 31000 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:36.061800 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "3fde787cfff9486cbb32544902fabd8e"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "7a771406a5010a2238c399d2d02f5685"
server_key_iv: "d6ca0261258eb55a2240d65f0119e1f6"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.062191 31000 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "3fde787cfff9486cbb32544902fabd8e"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "7a771406a5010a2238c399d2d02f5685"
server_key_iv: "d6ca0261258eb55a2240d65f0119e1f6"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.063530 31000 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:36.064378 31015 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.064567 31000 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:36.064669 31000 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "3fde787cfff9486cbb32544902fabd8e"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "7a771406a5010a2238c399d2d02f5685"
server_key_iv: "d6ca0261258eb55a2240d65f0119e1f6"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.064795 31000 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.075726 31000 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.076047 31000 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.076174 31000 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.081003 31000 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38865
I20260809 02:18:36.081125 31077 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38865 every 8 connection(s)
I20260809 02:18:36.081475 31000 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:36.082511 31078 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:36.084877 31078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: Bootstrap starting.
I20260809 02:18:36.085392 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31000
I20260809 02:18:36.085496 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:36.085532 31078 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:36.085741 9377 external_mini_cluster.cc:1428] Setting key 505d3e2c8f2b200812e9b3f8fa057caf
I20260809 02:18:36.085872 31078 log.cc:824] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:36.086715 31078 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: No bootstrap required, opened a new log
I20260809 02:18:36.089422 31078 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } }
I20260809 02:18:36.089570 31078 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:36.089592 31078 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3fde787cfff9486cbb32544902fabd8e, State: Initialized, Role: FOLLOWER
I20260809 02:18:36.089722 31078 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } }
I20260809 02:18:36.089794 31078 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:36.089833 31078 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:36.089881 31078 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:36.090643 31078 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } }
I20260809 02:18:36.090754 31078 leader_election.cc:302] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3fde787cfff9486cbb32544902fabd8e; no voters:
I20260809 02:18:36.090927 31078 leader_election.cc:288] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:36.091087 31084 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:36.091225 31078 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:36.091259 31084 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [term 1 LEADER]: Becoming Leader. State: Replica: 3fde787cfff9486cbb32544902fabd8e, State: Running, Role: LEADER
I20260809 02:18:36.091349 31084 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } }
I20260809 02:18:36.091717 31084 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3fde787cfff9486cbb32544902fabd8e. Latest consensus state: current_term: 1 leader_uuid: "3fde787cfff9486cbb32544902fabd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } } }
I20260809 02:18:36.091732 31086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3fde787cfff9486cbb32544902fabd8e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3fde787cfff9486cbb32544902fabd8e" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38865 } } }
I20260809 02:18:36.091827 31084 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:36.091846 31086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:36.092162 31092 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:36.092875 31092 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:36.094038 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.086378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48214 (local address 127.9.40.126:38865)
0809 02:18:36.086625 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.086634 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.086667 (+ 33us) server_negotiation.cc:407] Connection header received
0809 02:18:36.087200 (+ 533us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.087209 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.087287 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.087438 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:36.087626 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.089155 (+ 1529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.090245 (+ 1090us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.090894 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.091040 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.093249 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:36.093254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:36.093255 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:36.093457 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:36.093492 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:36.093503 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:36.093595 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.093726 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.093813 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":108,"threads_started":1}
I20260809 02:18:36.094571 31092 catalog_manager.cc:1347] Generated new cluster ID: cc7c5fed68fc4f2d843775a68d50ec4d
I20260809 02:18:36.094633 31092 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:36.114866 31092 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:36.115451 31092 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:36.125355 31092 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: Generated new TSK 0
I20260809 02:18:36.125684 31092 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:36.135661 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.65:0
--local_ip_for_outbound_sockets=127.9.40.65
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--builtin_ntp_servers=127.9.40.84:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:36.223001 31104 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.223201 31104 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.223222 31104 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.224363 31104 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.224406 31104 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.224431 31104 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.65
I20260809 02:18:36.225821 31104 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.9.40.65
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31104
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.65
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:36.226086 31104 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.226415 31104 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.231854 31112 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.231873 31110 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.231905 31104 server_base.cc:1034] running on GCE node
W20260809 02:18:36.231879 31109 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.232271 31104 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.232570 31104 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.233749 31104 hybrid_clock.cc:648] HybridClock initialized: now 1786241916233752 us; error 46 us; skew 500 ppm
I20260809 02:18:36.235149 31104 webserver.cc:460] Webserver started at http://127.9.40.65:46643/ using document root <none> and password file <none>
I20260809 02:18:36.235450 31104 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.235569 31104 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.235724 31104 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:36.236953 31104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data/instance:
uuid: "bea8b66b06ae489498171ce5f775cd00"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "f93141c04113290a46fd5c4a258ec6f7"
server_key_iv: "4699193e9f2e14bd68e4b4cc7b7171e2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.237331 31104 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance:
uuid: "bea8b66b06ae489498171ce5f775cd00"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "f93141c04113290a46fd5c4a258ec6f7"
server_key_iv: "4699193e9f2e14bd68e4b4cc7b7171e2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.238691 31104 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:36.239436 31120 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.239616 31104 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:36.239718 31104 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal
uuid: "bea8b66b06ae489498171ce5f775cd00"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "f93141c04113290a46fd5c4a258ec6f7"
server_key_iv: "4699193e9f2e14bd68e4b4cc7b7171e2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.239804 31104 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.255170 31104 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.255524 31104 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.255687 31104 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.255928 31104 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.256281 31104 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:36.256336 31104 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.256372 31104 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:36.256420 31104 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.262467 31104 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.65:33941
I20260809 02:18:36.262550 31244 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.65:33941 every 8 connection(s)
I20260809 02:18:36.262907 31104 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/data/info.pb
I20260809 02:18:36.268512 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.263677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.65:58539 (local address 127.9.40.126:38865)
0809 02:18:36.263748 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.263750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.264012 (+ 262us) server_negotiation.cc:407] Connection header received
0809 02:18:36.264574 (+ 562us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.264575 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.264587 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.264660 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:36.265504 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.266008 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.267010 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.267234 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.267327 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.268264 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:36.268266 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:36.268266 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:36.268271 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:36.268285 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:36.268289 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:36.268335 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.268423 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.268446 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260809 02:18:36.268793 31247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.263654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38865 (local address 127.9.40.65:58539)
0809 02:18:36.263872 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.263915 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.264455 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.264790 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.264795 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.264805 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:36.265400 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.265405 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.266134 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.266134 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.266941 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.266943 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.267360 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.267360 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.267525 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.268128 (+ 603us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:36.268129 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:36.268157 (+ 28us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:36.268181 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:36.268349 (+ 168us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:36.268354 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:36.268356 (+ 2us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.268411 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.268493 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":113,"threads_started":1}
I20260809 02:18:36.269447 31245 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38865
I20260809 02:18:36.269560 31245 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:36.269757 31245 heartbeater.cc:511] Master 127.9.40.126:38865 requested a full tablet report, sending...
I20260809 02:18:36.270332 31034 ts_manager.cc:194] Registered new tserver with Master: bea8b66b06ae489498171ce5f775cd00 (127.9.40.65:33941)
I20260809 02:18:36.270932 31034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.65:58539
I20260809 02:18:36.272505 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31104
I20260809 02:18:36.272600 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-0/wal/instance
I20260809 02:18:36.272823 9377 external_mini_cluster.cc:1428] Setting key d31b6bea6b3903206cd776600fa4ecdd
I20260809 02:18:36.273761 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.66:0
--local_ip_for_outbound_sockets=127.9.40.66
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--builtin_ntp_servers=127.9.40.84:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:36.357592 31248 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.357815 31248 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.357837 31248 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.359031 31248 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.359076 31248 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.359099 31248 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.66
I20260809 02:18:36.360412 31248 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.9.40.66
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31248
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.66
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:36.360606 31248 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.360878 31248 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.365927 31254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.365967 31256 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.366030 31248 server_base.cc:1034] running on GCE node
W20260809 02:18:36.365967 31253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.366511 31248 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.366792 31248 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.367981 31248 hybrid_clock.cc:648] HybridClock initialized: now 1786241916367959 us; error 36 us; skew 500 ppm
I20260809 02:18:36.369347 31248 webserver.cc:460] Webserver started at http://127.9.40.66:37163/ using document root <none> and password file <none>
I20260809 02:18:36.369637 31248 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.369709 31248 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.369871 31248 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:36.370980 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data/instance:
uuid: "db3e8b9b027740fe9a2274d46746adc5"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "b8a298d45c231b321111e4209d2a6aae"
server_key_iv: "bc28a13059f3194081537cd77bc831c0"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.371357 31248 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance:
uuid: "db3e8b9b027740fe9a2274d46746adc5"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "b8a298d45c231b321111e4209d2a6aae"
server_key_iv: "bc28a13059f3194081537cd77bc831c0"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.372661 31248 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:18:36.373421 31264 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.373601 31248 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:36.373705 31248 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal
uuid: "db3e8b9b027740fe9a2274d46746adc5"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "b8a298d45c231b321111e4209d2a6aae"
server_key_iv: "bc28a13059f3194081537cd77bc831c0"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.373780 31248 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.382615 31248 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.382937 31248 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.383100 31248 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.383335 31248 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.383673 31248 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:36.383730 31248 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.383772 31248 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:36.383800 31248 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.389808 31248 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.66:42277
I20260809 02:18:36.389885 31393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.66:42277 every 8 connection(s)
I20260809 02:18:36.390228 31248 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/data/info.pb
I20260809 02:18:36.395792 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.391054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:52063 (local address 127.9.40.126:38865)
0809 02:18:36.391141 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.391144 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.391394 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:18:36.392020 (+ 626us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.392021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.392033 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.392078 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:36.392809 (+ 731us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.393331 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.394335 (+ 1004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.394533 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.394618 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.395463 (+ 845us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:36.395464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:36.395465 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:36.395470 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:36.395493 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:36.395497 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:36.395579 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.395677 (+ 98us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.395708 (+ 31us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260809 02:18:36.396052 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.391041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38865 (local address 127.9.40.66:52063)
0809 02:18:36.391272 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.391307 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.391900 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.392164 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.392167 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.392175 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:36.392668 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.392672 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.393434 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.393435 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.394229 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.394232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.394626 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.394627 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.394759 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.395318 (+ 559us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:36.395318 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:36.395347 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:36.395370 (+ 23us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:36.395584 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:36.395591 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:36.395594 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.395679 (+ 85us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.395786 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":127,"threads_started":1}
I20260809 02:18:36.396596 31394 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38865
I20260809 02:18:36.396682 31394 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:36.396914 31394 heartbeater.cc:511] Master 127.9.40.126:38865 requested a full tablet report, sending...
I20260809 02:18:36.397310 31034 ts_manager.cc:194] Registered new tserver with Master: db3e8b9b027740fe9a2274d46746adc5 (127.9.40.66:42277)
I20260809 02:18:36.397697 31034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.66:52063
I20260809 02:18:36.399417 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31248
I20260809 02:18:36.399504 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-1/wal/instance
I20260809 02:18:36.399667 9377 external_mini_cluster.cc:1428] Setting key 9288b2fe760931183b3bce0ab7004084
I20260809 02:18:36.400570 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.9.40.67:0
--local_ip_for_outbound_sockets=127.9.40.67
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--builtin_ntp_servers=127.9.40.84:40935
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260809 02:18:36.484196 31397 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:36.484404 31397 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:36.484426 31397 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:36.485647 31397 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:36.485688 31397 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:36.485713 31397 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.9.40.67
I20260809 02:18:36.487015 31397 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:40935
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.9.40.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.9.40.67
--webserver_port=0
--tserver_master_addrs=127.9.40.126:38865
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31397
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.9.40.67
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:36.487215 31397 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:36.487468 31397 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:36.492592 31403 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:36.492602 31405 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.492648 31397 server_base.cc:1034] running on GCE node
W20260809 02:18:36.492895 31402 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:36.493150 31397 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:36.493356 31397 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:36.494499 31397 hybrid_clock.cc:648] HybridClock initialized: now 1786241916494468 us; error 50 us; skew 500 ppm
I20260809 02:18:36.495749 31397 webserver.cc:460] Webserver started at http://127.9.40.67:46481/ using document root <none> and password file <none>
I20260809 02:18:36.496002 31397 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:36.496062 31397 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:36.496235 31397 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:36.497493 31397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data/instance:
uuid: "2d21020a668642ee8888f3fa521370ca"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "4b3a47fdb9374824746c4f96e0ab087e"
server_key_iv: "c09c8ceaec346e3164148f0dec1327b2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.497860 31397 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance:
uuid: "2d21020a668642ee8888f3fa521370ca"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "4b3a47fdb9374824746c4f96e0ab087e"
server_key_iv: "c09c8ceaec346e3164148f0dec1327b2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.499161 31397 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:36.499877 31411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.500056 31397 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:36.500144 31397 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal
uuid: "2d21020a668642ee8888f3fa521370ca"
format_stamp: "Formatted at 2026-08-09 02:18:36 on dist-test-slave-qg60"
server_key: "4b3a47fdb9374824746c4f96e0ab087e"
server_key_iv: "c09c8ceaec346e3164148f0dec1327b2"
server_key_version: "encryptionkey@0"
I20260809 02:18:36.500216 31397 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:36.514906 31397 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:36.515245 31397 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:36.515403 31397 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:36.515650 31397 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260809 02:18:36.516054 31397 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260809 02:18:36.516119 31397 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.516166 31397 ts_tablet_manager.cc:610] Registered 0 tablets
I20260809 02:18:36.516194 31397 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:36.522166 31397 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.67:44753
I20260809 02:18:36.522249 31537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.67:44753 every 8 connection(s)
I20260809 02:18:36.522617 31397 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/data/info.pb
I20260809 02:18:36.526223 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31397
I20260809 02:18:36.526352 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal/instance
I20260809 02:18:36.526561 9377 external_mini_cluster.cc:1428] Setting key 61106dd7931d620e5e4665bcca812254
I20260809 02:18:36.528352 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.523526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.67:38193 (local address 127.9.40.126:38865)
0809 02:18:36.523575 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.523577 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.523829 (+ 252us) server_negotiation.cc:407] Connection header received
0809 02:18:36.524306 (+ 477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.524307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.524318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.524384 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:36.525213 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.525702 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.526800 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.527066 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.527139 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.528110 (+ 971us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:36.528111 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:36.528112 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:36.528116 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:36.528130 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:36.528133 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:36.528179 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.528239 (+ 60us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.528266 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:18:36.528545 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.523473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38865 (local address 127.9.40.67:38193)
0809 02:18:36.523693 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.523735 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.524217 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.524449 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.524453 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.524460 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:18:36.525093 (+ 633us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.525099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.525785 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.525785 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.526714 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.526716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.527161 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.527162 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.527321 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.527959 (+ 638us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0809 02:18:36.527959 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:18:36.527988 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0809 02:18:36.528010 (+ 22us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:18:36.528190 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:18:36.528192 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:18:36.528195 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.528240 (+ 45us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.528323 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":99,"threads_started":1}
I20260809 02:18:36.529033 31538 heartbeater.cc:347] Connected to a master server at 127.9.40.126:38865
I20260809 02:18:36.529137 31538 heartbeater.cc:464] Registering TS with master...
I20260809 02:18:36.529311 31538 heartbeater.cc:511] Master 127.9.40.126:38865 requested a full tablet report, sending...
I20260809 02:18:36.529665 31034 ts_manager.cc:194] Registered new tserver with Master: 2d21020a668642ee8888f3fa521370ca (127.9.40.67:44753)
I20260809 02:18:36.530045 31034 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.9.40.67:38193
I20260809 02:18:36.538583 9377 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260809 02:18:36.543144 31083 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.540022 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48226 (local address 127.9.40.126:38865)
0809 02:18:36.540090 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.540093 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.540199 (+ 106us) server_negotiation.cc:407] Connection header received
0809 02:18:36.540248 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.540249 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.540259 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.540312 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:36.540442 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.541030 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.541793 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.542059 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.542111 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.542843 (+ 732us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:36.542844 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:36.542844 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:36.542848 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:36.542863 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:36.542866 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:36.542910 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.542988 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.543033 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260809 02:18:36.544695 31034 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:48226:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260809 02:18:36.546499 31034 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260809 02:18:36.553258 31547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.548822 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42277 (local address 127.0.0.1:46522)
0809 02:18:36.549134 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.549180 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.549268 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.549690 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.549693 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.549696 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:36.549769 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.549771 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.551127 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.551128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.552196 (+ 1068us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.552198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.552939 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.552940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.553022 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553036 (+ 14us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.553120 (+ 84us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.553175 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":9,"thread_start_us":69,"threads_started":1}
I20260809 02:18:36.553270 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.548941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46522 (local address 127.9.40.66:42277)
0809 02:18:36.549192 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.549196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.549241 (+ 45us) server_negotiation.cc:407] Connection header received
0809 02:18:36.549397 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.549401 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.549420 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.549552 (+ 132us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:36.549835 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.551022 (+ 1187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.552264 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.552841 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.552925 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553030 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.553137 (+ 107us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.553175 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":203,"thread_start_us":142,"threads_started":1}
I20260809 02:18:36.553457 31546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.549194 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33941 (local address 127.0.0.1:52104)
0809 02:18:36.549395 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.549406 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.549469 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.549781 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.549782 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.549784 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:36.549818 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.549819 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.550817 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.550818 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.552108 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.552110 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.553190 (+ 1080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.553191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.553252 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553256 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.553311 (+ 55us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.553352 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":79,"threads_started":1}
I20260809 02:18:36.553965 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.549422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52104 (local address 127.9.40.65:33941)
0809 02:18:36.549634 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.549638 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.549649 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:18:36.549689 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.549691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.549705 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.549773 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:36.549880 (+ 107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.550723 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.552212 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.552977 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.553052 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553148 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.553631 (+ 483us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.553675 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179,"thread_start_us":90,"threads_started":1}
I20260809 02:18:36.554037 31322 tablet_service.cc:1467] Processing CreateTablet for tablet de796d47b1544129af338a335942a353 (DEFAULT_TABLE table=test-table [id=bfefc38330844ba59abb9d5471174ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:36.554200 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.548849 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44753 (local address 127.0.0.1:38100)
0809 02:18:36.549313 (+ 464us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.549326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.549380 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.550097 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.550098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.550100 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:36.550137 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.550138 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.551506 (+ 1368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.551507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.552435 (+ 928us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.552438 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.553785 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.553786 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.553859 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553863 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.553914 (+ 51us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.553954 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":206,"threads_started":1}
I20260809 02:18:36.554459 31322 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de796d47b1544129af338a335942a353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:36.554658 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.549198 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38100 (local address 127.9.40.67:44753)
0809 02:18:36.549912 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.549926 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.549936 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:18:36.549972 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.549973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.549987 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.550073 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:36.550203 (+ 130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.551339 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.552563 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.553475 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.553657 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.553754 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.554493 (+ 739us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.554577 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":668,"thread_start_us":94,"threads_started":1}
I20260809 02:18:36.554621 31174 tablet_service.cc:1467] Processing CreateTablet for tablet de796d47b1544129af338a335942a353 (DEFAULT_TABLE table=test-table [id=bfefc38330844ba59abb9d5471174ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:36.554922 31174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de796d47b1544129af338a335942a353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:36.557255 31465 tablet_service.cc:1467] Processing CreateTablet for tablet de796d47b1544129af338a335942a353 (DEFAULT_TABLE table=test-table [id=bfefc38330844ba59abb9d5471174ed9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260809 02:18:36.557583 31465 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet de796d47b1544129af338a335942a353. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:36.558172 31553 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Bootstrap starting.
I20260809 02:18:36.558401 31555 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: Bootstrap starting.
I20260809 02:18:36.558907 31553 tablet_bootstrap.cc:654] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:36.559001 31555 tablet_bootstrap.cc:654] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:36.559264 31553 log.cc:824] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:36.559289 31555 log.cc:824] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:36.560148 31555 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: No bootstrap required, opened a new log
I20260809 02:18:36.560220 31555 ts_tablet_manager.cc:1397] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:36.561039 31558 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Bootstrap starting.
I20260809 02:18:36.561175 31553 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: No bootstrap required, opened a new log
I20260809 02:18:36.561245 31553 ts_tablet_manager.cc:1397] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260809 02:18:36.561705 31558 tablet_bootstrap.cc:654] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:36.562052 31558 log.cc:824] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:36.562659 31555 raft_consensus.cc:348] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.562811 31555 raft_consensus.cc:374] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:36.562855 31555 raft_consensus.cc:729] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea8b66b06ae489498171ce5f775cd00, State: Initialized, Role: FOLLOWER
I20260809 02:18:36.562887 31558 tablet_bootstrap.cc:492] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: No bootstrap required, opened a new log
I20260809 02:18:36.562954 31558 ts_tablet_manager.cc:1397] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260809 02:18:36.562974 31555 consensus_queue.cc:260] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.563283 31555 ts_tablet_manager.cc:1428] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:36.563313 31245 heartbeater.cc:503] Master 127.9.40.126:38865 was elected leader, sending a full tablet report...
I20260809 02:18:36.564055 31553 raft_consensus.cc:348] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.564214 31553 raft_consensus.cc:374] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:36.564260 31553 raft_consensus.cc:729] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3e8b9b027740fe9a2274d46746adc5, State: Initialized, Role: FOLLOWER
I20260809 02:18:36.564393 31553 consensus_queue.cc:260] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.564664 31553 ts_tablet_manager.cc:1428] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:36.564803 31394 heartbeater.cc:503] Master 127.9.40.126:38865 was elected leader, sending a full tablet report...
I20260809 02:18:36.565634 31558 raft_consensus.cc:348] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.565748 31558 raft_consensus.cc:374] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:36.565769 31558 raft_consensus.cc:729] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d21020a668642ee8888f3fa521370ca, State: Initialized, Role: FOLLOWER
I20260809 02:18:36.565876 31558 consensus_queue.cc:260] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.566123 31558 ts_tablet_manager.cc:1428] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260809 02:18:36.566159 31538 heartbeater.cc:503] Master 127.9.40.126:38865 was elected leader, sending a full tablet report...
I20260809 02:18:36.583845 31561 raft_consensus.cc:482] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260809 02:18:36.583974 31561 raft_consensus.cc:504] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.584448 31561 leader_election.cc:288] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2d21020a668642ee8888f3fa521370ca (127.9.40.67:44753), bea8b66b06ae489498171ce5f775cd00 (127.9.40.65:33941)
I20260809 02:18:36.587296 31550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.584773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:59447 (local address 127.9.40.67:44753)
0809 02:18:36.584816 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.584818 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.584830 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:18:36.584866 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.584867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.584878 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.584936 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:36.585157 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.585652 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.586587 (+ 935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.586951 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.587037 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.587047 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.587200 (+ 153us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.587238 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:18:36.587296 31396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.584664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.67:44753 (local address 127.9.40.66:59447)
0809 02:18:36.584720 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.584748 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.584793 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.584957 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.584958 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.584960 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:36.585076 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.585078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.585740 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.585741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.586492 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.586494 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.587037 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.587038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.587108 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.587111 (+ 3us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.587181 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.587222 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:18:36.587600 31551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.584910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.9.40.66:35575 (local address 127.9.40.65:33941)
0809 02:18:36.584960 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.584963 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.585081 (+ 118us) server_negotiation.cc:407] Connection header received
0809 02:18:36.585112 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.585113 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.585124 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.585169 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0809 02:18:36.585330 (+ 161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.585790 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.586798 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.587176 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.587271 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.587281 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.587470 (+ 189us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.587517 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:18:36.587664 31487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de796d47b1544129af338a335942a353" candidate_uuid: "db3e8b9b027740fe9a2274d46746adc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d21020a668642ee8888f3fa521370ca" is_pre_election: true
I20260809 02:18:36.587801 31563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.584806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33941 (local address 127.9.40.66:35575)
0809 02:18:36.584992 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.585001 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:18:36.585055 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:18:36.585180 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:18:36.585181 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:18:36.585182 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0809 02:18:36.585252 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.585254 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.585874 (+ 620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.585875 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.586707 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:18:36.586709 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.587283 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.587284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:18:36.587367 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.587371 (+ 4us) client_negotiation.cc:769] Sending connection context
0809 02:18:36.587692 (+ 321us) client_negotiation.cc:240] Negotiation successful
0809 02:18:36.587730 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":172,"thread_start_us":123,"threads_started":1}
I20260809 02:18:36.587810 31487 raft_consensus.cc:2454] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e8b9b027740fe9a2274d46746adc5 in term 0.
I20260809 02:18:36.588027 31267 leader_election.cc:302] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2d21020a668642ee8888f3fa521370ca, db3e8b9b027740fe9a2274d46746adc5; no voters:
I20260809 02:18:36.588150 31561 raft_consensus.cc:2781] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260809 02:18:36.588215 31561 raft_consensus.cc:482] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260809 02:18:36.588250 31561 raft_consensus.cc:3037] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:36.588435 31195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de796d47b1544129af338a335942a353" candidate_uuid: "db3e8b9b027740fe9a2274d46746adc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea8b66b06ae489498171ce5f775cd00" is_pre_election: true
I20260809 02:18:36.588579 31195 raft_consensus.cc:2454] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate db3e8b9b027740fe9a2274d46746adc5 in term 0.
I20260809 02:18:36.589207 31561 raft_consensus.cc:504] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.589342 31561 leader_election.cc:288] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [CANDIDATE]: Term 1 election: Requested vote from peers 2d21020a668642ee8888f3fa521370ca (127.9.40.67:44753), bea8b66b06ae489498171ce5f775cd00 (127.9.40.65:33941)
I20260809 02:18:36.589493 31487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de796d47b1544129af338a335942a353" candidate_uuid: "db3e8b9b027740fe9a2274d46746adc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2d21020a668642ee8888f3fa521370ca"
I20260809 02:18:36.589532 31195 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "de796d47b1544129af338a335942a353" candidate_uuid: "db3e8b9b027740fe9a2274d46746adc5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bea8b66b06ae489498171ce5f775cd00"
I20260809 02:18:36.589600 31487 raft_consensus.cc:3037] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:36.589627 31195 raft_consensus.cc:3037] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:36.590363 31195 raft_consensus.cc:2454] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e8b9b027740fe9a2274d46746adc5 in term 1.
I20260809 02:18:36.590386 31487 raft_consensus.cc:2454] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate db3e8b9b027740fe9a2274d46746adc5 in term 1.
I20260809 02:18:36.590540 31267 leader_election.cc:302] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bea8b66b06ae489498171ce5f775cd00, db3e8b9b027740fe9a2274d46746adc5; no voters:
I20260809 02:18:36.590663 31561 raft_consensus.cc:2781] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:36.590852 31561 raft_consensus.cc:686] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 1 LEADER]: Becoming Leader. State: Replica: db3e8b9b027740fe9a2274d46746adc5, State: Running, Role: LEADER
I20260809 02:18:36.590946 31561 consensus_queue.cc:237] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } }
I20260809 02:18:36.591799 31033 catalog_manager.cc:5495] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 reported cstate change: term changed from 0 to 1, leader changed from <none> to db3e8b9b027740fe9a2274d46746adc5 (127.9.40.66). New cstate: current_term: 1 leader_uuid: "db3e8b9b027740fe9a2274d46746adc5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3e8b9b027740fe9a2274d46746adc5" member_type: VOTER last_known_addr { host: "127.9.40.66" port: 42277 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 } health_report { overall_health: UNKNOWN } } }
I20260809 02:18:36.598261 31548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.596012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.9.40.66:42277)
0809 02:18:36.596058 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:36.596060 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:36.596097 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:18:36.596134 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:36.596134 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:36.596151 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:36.596206 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:18:36.596303 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.596844 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.597596 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:36.597788 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:36.597842 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:36.597969 (+ 127us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:18:36.598071 (+ 102us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:18:36.598133 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:36.598165 (+ 32us) server_negotiation.cc:299] Negotiation successful
0809 02:18:36.598181 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:18:36.600266 31195 raft_consensus.cc:1261] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 1 FOLLOWER]: Refusing update from remote peer db3e8b9b027740fe9a2274d46746adc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:36.600281 31487 raft_consensus.cc:1261] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 1 FOLLOWER]: Refusing update from remote peer db3e8b9b027740fe9a2274d46746adc5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:18:36.600680 31561 consensus_queue.cc:1035] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "bea8b66b06ae489498171ce5f775cd00" member_type: VOTER last_known_addr { host: "127.9.40.65" port: 33941 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:36.600868 31564 consensus_queue.cc:1035] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2d21020a668642ee8888f3fa521370ca" member_type: VOTER last_known_addr { host: "127.9.40.67" port: 44753 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:18:36.603376 31567 mvcc.cc:205] Tried to move back new op lower bound from 7316446890391097344 to 7316446890357264384. Current Snapshot: MvccSnapshot[applied={T|T < 7316446890391097344}]
I20260809 02:18:36.604220 31569 mvcc.cc:205] Tried to move back new op lower bound from 7316446890391097344 to 7316446890357264384. Current Snapshot: MvccSnapshot[applied={T|T < 7316446890391097344}]
I20260809 02:18:36.605927 31034 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:48226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:36.606034 31034 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:48226:
table { table_name: "test-table" } modify_external_catalogs: true
I20260809 02:18:36.606853 31034 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3fde787cfff9486cbb32544902fabd8e: Sending DeleteTablet for 3 replicas of tablet de796d47b1544129af338a335942a353
I20260809 02:18:36.607215 31465 tablet_service.cc:1514] Processing DeleteTablet for tablet de796d47b1544129af338a335942a353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:36 UTC) from {username='slave'} at 127.0.0.1:38100
I20260809 02:18:36.607255 31174 tablet_service.cc:1514] Processing DeleteTablet for tablet de796d47b1544129af338a335942a353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:36 UTC) from {username='slave'} at 127.0.0.1:52104
I20260809 02:18:36.607254 31322 tablet_service.cc:1514] Processing DeleteTablet for tablet de796d47b1544129af338a335942a353 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-09 02:18:36 UTC) from {username='slave'} at 127.0.0.1:46522
I20260809 02:18:36.607453 31578 tablet_replica.cc:330] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00: stopping tablet replica
I20260809 02:18:36.607585 31578 raft_consensus.cc:2229] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:36.607697 31578 raft_consensus.cc:2258] T de796d47b1544129af338a335942a353 P bea8b66b06ae489498171ce5f775cd00 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:36.607786 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31104
I20260809 02:18:36.608491 31577 tablet_replica.cc:330] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: stopping tablet replica
I20260809 02:18:36.608632 31577 raft_consensus.cc:2229] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260809 02:18:36.608795 31577 raft_consensus.cc:2258] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:36.609284 31580 tablet_replica.cc:330] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: stopping tablet replica
I20260809 02:18:36.609396 31580 raft_consensus.cc:2229] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 1 LEADER]: Raft consensus shutting down.
I20260809 02:18:36.609509 31580 raft_consensus.cc:2258] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260809 02:18:36.609931 31577 ts_tablet_manager.cc:1905] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:36.612165 31577 ts_tablet_manager.cc:1918] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260809 02:18:36.612248 31577 log.cc:1196] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Deleting WAL directory at /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786241669736972-9377-0/minicluster-data/ts-2/wal/wals/de796d47b1544129af338a335942a353
I20260809 02:18:36.612533 31577 ts_tablet_manager.cc:1939] T de796d47b1544129af338a335942a353 P 2d21020a668642ee8888f3fa521370ca: Deleting consensus metadata
I20260809 02:18:36.613212 31019 catalog_manager.cc:4841] TS 2d21020a668642ee8888f3fa521370ca (127.9.40.67:44753): tablet de796d47b1544129af338a335942a353 (table test-table [id=bfefc38330844ba59abb9d5471174ed9]) successfully deleted
W20260809 02:18:36.617106 31019 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.9.40.65:33941 (error 108)
I20260809 02:18:36.617120 31580 ts_tablet_manager.cc:1905] T de796d47b1544129af338a335942a353 P db3e8b9b027740fe9a2274d46746adc5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260809 02:18:36.617693 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31248
I20260809 02:18:36.617868 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.617731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.65:33941 (local address 127.0.0.1:52114)
0809 02:18:36.617786 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.617826 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:18:36.618870 31019 catalog_manager.cc:4584] TS bea8b66b06ae489498171ce5f775cd00 (127.9.40.65:33941): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet de796d47b1544129af338a335942a353: Network error: Client connection negotiation failed: client connection to 127.9.40.65:33941: connect: Connection refused (error 111)
I20260809 02:18:36.624869 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31397
I20260809 02:18:36.625010 31549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:36.624842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.66:42277 (local address 127.0.0.1:46534)
0809 02:18:36.624915 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0809 02:18:36.624966 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.66:42277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:18:36.630965 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31000
2026-08-09T02:18:36Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (711 ms)
[----------] 6 tests from EncryptionPolicyTest (9029 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:36 dist-test-slave-qg60 krb5kdc[31584](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:36 dist-test-slave-qg60 krb5kdc[31584](info): set up 2 sockets
Aug 09 02:18:36 dist-test-slave-qg60 krb5kdc[31584](info): commencing operation
krb5kdc: starting...
W20260809 02:18:37.729269 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:37 dist-test-slave-qg60 krb5kdc[31584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:37.863759 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36351
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36351
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:37.948786 31600 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:37.949084 31600 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:37.949126 31600 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:37.950302 31600 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:37.950377 31600 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:37.950402 31600 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:37.950424 31600 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:37.950443 31600 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:37.952031 31600 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36351
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36351
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31600
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:37.952337 31600 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:37.952606 31600 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:37.957916 31606 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:37.957943 31607 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:37.958210 31600 server_base.cc:1034] running on GCE node
W20260809 02:18:37.957916 31609 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:37.958536 31600 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:37.958904 31600 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:37.960083 31600 hybrid_clock.cc:648] HybridClock initialized: now 1786241917960060 us; error 38 us; skew 500 ppm
Aug 09 02:18:37 dist-test-slave-qg60 krb5kdc[31584](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241917, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:37.968633 31600 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:37.969187 31600 webserver.cc:460] Webserver started at http://127.0.0.1:38937/ using document root <none> and password file <none>
I20260809 02:18:37.969468 31600 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:37.969542 31600 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:37.969702 31600 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:37.970827 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "def7d9bffac44161b8ab55ea8892898e"
format_stamp: "Formatted at 2026-08-09 02:18:37 on dist-test-slave-qg60"
server_key: "2449bfc2e6a59eb6d3690c7ee14c6506"
server_key_iv: "ec1d0e12c3fcfaa224ca6b39c69c6f1d"
server_key_version: "encryptionkey@0"
I20260809 02:18:37.971220 31600 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "def7d9bffac44161b8ab55ea8892898e"
format_stamp: "Formatted at 2026-08-09 02:18:37 on dist-test-slave-qg60"
server_key: "2449bfc2e6a59eb6d3690c7ee14c6506"
server_key_iv: "ec1d0e12c3fcfaa224ca6b39c69c6f1d"
server_key_version: "encryptionkey@0"
I20260809 02:18:37.972710 31600 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:37.973677 31618 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:37.974030 31600 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:37.974146 31600 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "def7d9bffac44161b8ab55ea8892898e"
format_stamp: "Formatted at 2026-08-09 02:18:37 on dist-test-slave-qg60"
server_key: "2449bfc2e6a59eb6d3690c7ee14c6506"
server_key_iv: "ec1d0e12c3fcfaa224ca6b39c69c6f1d"
server_key_version: "encryptionkey@0"
I20260809 02:18:37.974242 31600 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:37.983448 31600 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:37.985626 31600 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:37.985797 31600 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:37.990242 31600 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36351
I20260809 02:18:37.990326 31674 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36351 every 8 connection(s)
I20260809 02:18:37.990687 31600 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:37.991778 31675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:37.994210 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e: Bootstrap starting.
I20260809 02:18:37.994802 31675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:37.995065 31675 log.cc:824] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:37.995820 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e: No bootstrap required, opened a new log
I20260809 02:18:37.997623 31675 raft_consensus.cc:348] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 02:18:37.997769 31675 raft_consensus.cc:374] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:37.997795 31675 raft_consensus.cc:729] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: def7d9bffac44161b8ab55ea8892898e, State: Initialized, Role: FOLLOWER
I20260809 02:18:37.997937 31675 consensus_queue.cc:260] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 02:18:37.998020 31675 raft_consensus.cc:388] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:37.998044 31675 raft_consensus.cc:482] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:37.998075 31675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:37.998586 31675 raft_consensus.cc:504] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 02:18:37.998677 31675 leader_election.cc:302] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: def7d9bffac44161b8ab55ea8892898e; no voters:
I20260809 02:18:37.998818 31675 leader_election.cc:288] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:37.998898 31678 raft_consensus.cc:2781] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:37.999089 31678 raft_consensus.cc:686] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [term 1 LEADER]: Becoming Leader. State: Replica: def7d9bffac44161b8ab55ea8892898e, State: Running, Role: LEADER
I20260809 02:18:37.999137 31675 sys_catalog.cc:564] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:37.999179 31678 consensus_queue.cc:237] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } }
I20260809 02:18:37.999619 31681 sys_catalog.cc:455] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [sys.catalog]: SysCatalogTable state changed. Reason: New leader def7d9bffac44161b8ab55ea8892898e. Latest consensus state: current_term: 1 leader_uuid: "def7d9bffac44161b8ab55ea8892898e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } } }
I20260809 02:18:37.999714 31681 sys_catalog.cc:458] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:37.999758 31680 sys_catalog.cc:455] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "def7d9bffac44161b8ab55ea8892898e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "def7d9bffac44161b8ab55ea8892898e" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36351 } } }
I20260809 02:18:37.999816 31680 sys_catalog.cc:458] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:37.999984 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31600
I20260809 02:18:37.999958 31688 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:38.000088 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:38.000317 9377 external_mini_cluster.cc:1428] Setting key 0e6395e8cc8fb49cf9432654cb664f2c
I20260809 02:18:38.000546 31688 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:38.002601 31688 catalog_manager.cc:1347] Generated new cluster ID: f9356ba44a6e4faa8e2f86ace3f54251
I20260809 02:18:38.002660 31688 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:38 dist-test-slave-qg60 krb5kdc[31584](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:18:38.011554 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:38.000942 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56563 (local address 127.0.0.1:36351)
0809 02:18:38.001146 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:38.001156 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:38.001775 (+ 619us) server_negotiation.cc:407] Connection header received
0809 02:18:38.001941 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:38.001948 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:38.002008 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:38.002172 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:38.003050 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:38.003994 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:38.004784 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:38.005278 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:38.005401 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:38.007908 (+ 2507us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:38.007919 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:38.007919 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:38.007931 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:38.009806 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:38.010222 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:38.010226 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:38.010228 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:38.010332 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:38.010585 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:38.010586 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:38.010587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:38.010842 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:38.011044 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:38.011135 (+ 91us) server_negotiation.cc:299] Negotiation successful
0809 02:18:38.011350 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":106,"threads_started":1}
I20260809 02:18:38.012605 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:38.023780 31696 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:38.014138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47881 (local address 127.0.0.1:36351)
0809 02:18:38.014181 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:38.014183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:38.017059 (+ 2876us) server_negotiation.cc:407] Connection header received
0809 02:18:38.017146 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:38.017147 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:38.017159 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:38.017206 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:38.017965 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:38.018505 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:38.019238 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:38.019470 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:38.019552 (+ 82us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:38.020919 (+ 1367us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:38.020920 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:38.020921 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:38.020929 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:38.022457 (+ 1528us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:38.022902 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:38.022903 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:38.022904 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:38.022974 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:38.023214 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:38.023215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:38.023215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:38.023342 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:38.023490 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:38.023591 (+ 101us) server_negotiation.cc:299] Negotiation successful
0809 02:18:38.023661 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:18:38.026427 31688 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:38.027222 31688 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:38.030205 31688 catalog_manager.cc:5868] T 00000000000000000000000000000000 P def7d9bffac44161b8ab55ea8892898e: Generated new TSK 0
I20260809 02:18:38.030398 31688 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:38.036213 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31600
W20260809 02:18:38.043479 31582 connection.cc:535] client connection to 127.0.0.1:36351 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:36351): Connection reset by peer (error 104)
2026-08-09T02:18:38Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1404 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:38 dist-test-slave-qg60 krb5kdc[31707](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:38 dist-test-slave-qg60 krb5kdc[31707](info): set up 2 sockets
Aug 09 02:18:38 dist-test-slave-qg60 krb5kdc[31707](info): commencing operation
krb5kdc: starting...
W20260809 02:18:39.158553 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.101s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:39 dist-test-slave-qg60 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241919, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:39.293236 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43701
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43701
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:39.378064 31721 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:39.378288 31721 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:39.378312 31721 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:39.379498 31721 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:39.379577 31721 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:39.379593 31721 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:39.379647 31721 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:39.379665 31721 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:39.381263 31721 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43701
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31721
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:39.381531 31721 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:39.381783 31721 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:39.386842 31726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:39.386847 31730 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:39.386864 31727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:39.386871 31721 server_base.cc:1034] running on GCE node
I20260809 02:18:39.387382 31721 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:39.387678 31721 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:39.388803 31721 hybrid_clock.cc:648] HybridClock initialized: now 1786241919388785 us; error 36 us; skew 500 ppm
Aug 09 02:18:39 dist-test-slave-qg60 krb5kdc[31707](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241919, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:39.391363 31721 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:39.391811 31721 webserver.cc:460] Webserver started at http://127.0.0.1:45875/ using document root <none> and password file <none>
I20260809 02:18:39.392058 31721 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:39.392131 31721 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:39.392283 31721 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:39.393458 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "2ab71c1af8f343bd894c6bf10d88afd4"
format_stamp: "Formatted at 2026-08-09 02:18:39 on dist-test-slave-qg60"
server_key: "f44f6e7ed0cad7c75115bf3fe0fc67bc"
server_key_iv: "a6d28ddf1aa54751095ccb102a4a15fa"
server_key_version: "encryptionkey@0"
I20260809 02:18:39.393842 31721 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2ab71c1af8f343bd894c6bf10d88afd4"
format_stamp: "Formatted at 2026-08-09 02:18:39 on dist-test-slave-qg60"
server_key: "f44f6e7ed0cad7c75115bf3fe0fc67bc"
server_key_iv: "a6d28ddf1aa54751095ccb102a4a15fa"
server_key_version: "encryptionkey@0"
I20260809 02:18:39.395277 31721 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:39.396024 31737 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:39.396224 31721 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:18:39.396332 31721 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "2ab71c1af8f343bd894c6bf10d88afd4"
format_stamp: "Formatted at 2026-08-09 02:18:39 on dist-test-slave-qg60"
server_key: "f44f6e7ed0cad7c75115bf3fe0fc67bc"
server_key_iv: "a6d28ddf1aa54751095ccb102a4a15fa"
server_key_version: "encryptionkey@0"
I20260809 02:18:39.396415 31721 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:39.410149 31721 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:39.412094 31721 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:39.412272 31721 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:39.416438 31721 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43701
I20260809 02:18:39.416493 31793 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43701 every 8 connection(s)
I20260809 02:18:39.416879 31721 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:39.417917 31794 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:39.419086 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31721
I20260809 02:18:39.419229 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:39.419437 9377 external_mini_cluster.cc:1428] Setting key de654454fae0fded7b3f9515cad64d96
I20260809 02:18:39.420619 31794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4: Bootstrap starting.
I20260809 02:18:39.421248 31794 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:39.421553 31794 log.cc:824] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:39.422262 31794 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4: No bootstrap required, opened a new log
I20260809 02:18:39.424681 31794 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260809 02:18:39.424903 31794 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:39.424930 31794 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ab71c1af8f343bd894c6bf10d88afd4, State: Initialized, Role: FOLLOWER
I20260809 02:18:39.425081 31794 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260809 02:18:39.425189 31794 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:39.425230 31794 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:39.425288 31794 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:39.425534 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:39.420000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49433 (local address 127.0.0.1:43701)
0809 02:18:39.420221 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:39.420231 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:39.420270 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:18:39.420712 (+ 442us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:39.420722 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:39.420805 (+ 83us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:39.420956 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:39.421209 (+ 253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:39.422268 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:39.423419 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:39.423822 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:39.423902 (+ 80us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:39.424717 (+ 815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:39.424721 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:39.424721 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:39.424994 (+ 273us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:39.425039 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:39.425060 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:39.425148 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:39.425309 (+ 161us) server_negotiation.cc:299] Negotiation successful
0809 02:18:39.425341 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":88,"threads_started":1}
I20260809 02:18:39.425949 31794 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260809 02:18:39.426102 31794 leader_election.cc:302] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ab71c1af8f343bd894c6bf10d88afd4; no voters:
I20260809 02:18:39.426297 31794 leader_election.cc:288] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:39.426321 31800 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:39.426432 31800 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [term 1 LEADER]: Becoming Leader. State: Replica: 2ab71c1af8f343bd894c6bf10d88afd4, State: Running, Role: LEADER
I20260809 02:18:39.426534 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:39.426548 31794 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:39.427685 31800 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } }
I20260809 02:18:39.428364 31802 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ab71c1af8f343bd894c6bf10d88afd4. Latest consensus state: current_term: 1 leader_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } } }
I20260809 02:18:39.428474 31802 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:39.428373 31801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ab71c1af8f343bd894c6bf10d88afd4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43701 } } }
I20260809 02:18:39.428690 31801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:39.428858 31815 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:39.429414 31815 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:39.431049 31815 catalog_manager.cc:1347] Generated new cluster ID: 81d46b9ed69b48c1ae5841cd9880f3b1
I20260809 02:18:39.431113 31815 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:39.432683 31798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:39.428018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44741 (local address 127.0.0.1:43701)
0809 02:18:39.428079 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:39.428081 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:39.429124 (+ 1043us) server_negotiation.cc:407] Connection header received
0809 02:18:39.429159 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:39.429160 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:39.429172 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:39.429239 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:39.429451 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:39.430202 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:39.431322 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:39.431582 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:39.431665 (+ 83us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:39.432388 (+ 723us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:39.432390 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:39.432390 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:39.432395 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:39.432411 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:39.432415 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:39.432493 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:39.432577 (+ 84us) server_negotiation.cc:299] Negotiation successful
0809 02:18:39.432588 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:18:39.437180 31815 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:39.437892 31815 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:39.441532 31815 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ab71c1af8f343bd894c6bf10d88afd4: Generated new TSK 0
I20260809 02:18:39.441706 31815 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:39.446148 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31721
W20260809 02:18:39.452999 31705 connection.cc:535] client connection to 127.0.0.1:43701 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43701): Connection reset by peer (error 104)
2026-08-09T02:18:39Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1409 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:39 dist-test-slave-qg60 krb5kdc[31833](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:39 dist-test-slave-qg60 krb5kdc[31833](info): set up 2 sockets
Aug 09 02:18:39 dist-test-slave-qg60 krb5kdc[31833](info): commencing operation
krb5kdc: starting...
W20260809 02:18:40.537976 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:40 dist-test-slave-qg60 krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:40.672350 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45381
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:40.765934 31847 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:40.766161 31847 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:40.766186 31847 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:40.767401 31847 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:40.767449 31847 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:40.767463 31847 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:40.767477 31847 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:40.767488 31847 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:40.769006 31847 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45381
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31847
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:40.769210 31847 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:40.769385 31847 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:40.774636 31854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:40.774609 31856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:40.774609 31853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:40.775015 31847 server_base.cc:1034] running on GCE node
I20260809 02:18:40.775274 31847 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:40.775599 31847 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:40.776789 31847 hybrid_clock.cc:648] HybridClock initialized: now 1786241920776776 us; error 28 us; skew 500 ppm
Aug 09 02:18:40 dist-test-slave-qg60 krb5kdc[31833](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241920, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:40.779428 31847 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:40.779939 31847 webserver.cc:460] Webserver started at http://127.0.0.1:33037/ using document root <none> and password file <none>
I20260809 02:18:40.780175 31847 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:40.780244 31847 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:40.780400 31847 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:40.781562 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "b6fdcfd8431e4144aa342184d85d20a1"
format_stamp: "Formatted at 2026-08-09 02:18:40 on dist-test-slave-qg60"
server_key: "ab63e7d6910fd60e70800f306c7f23be"
server_key_iv: "bd9c701b2676bac18181aa52e091b14f"
server_key_version: "encryptionkey@0"
I20260809 02:18:40.781945 31847 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "b6fdcfd8431e4144aa342184d85d20a1"
format_stamp: "Formatted at 2026-08-09 02:18:40 on dist-test-slave-qg60"
server_key: "ab63e7d6910fd60e70800f306c7f23be"
server_key_iv: "bd9c701b2676bac18181aa52e091b14f"
server_key_version: "encryptionkey@0"
I20260809 02:18:40.783354 31847 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:40.784204 31864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:40.784453 31847 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:40.784626 31847 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "b6fdcfd8431e4144aa342184d85d20a1"
format_stamp: "Formatted at 2026-08-09 02:18:40 on dist-test-slave-qg60"
server_key: "ab63e7d6910fd60e70800f306c7f23be"
server_key_iv: "bd9c701b2676bac18181aa52e091b14f"
server_key_version: "encryptionkey@0"
I20260809 02:18:40.784704 31847 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:40.788982 31847 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:40.791311 31847 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:40.791476 31847 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:40.795786 31847 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45381
I20260809 02:18:40.795817 31922 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45381 every 8 connection(s)
I20260809 02:18:40.796152 31847 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:40.797253 31923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:40.798238 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31847
I20260809 02:18:40.798367 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:40.798594 9377 external_mini_cluster.cc:1428] Setting key 8149cdfcbb25fc245aaa251a46550994
I20260809 02:18:40.800707 31923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1: Bootstrap starting.
I20260809 02:18:40.801072 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:40.799127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59757 (local address 127.0.0.1:45381)
0809 02:18:40.799315 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:40.799322 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:40.799348 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:18:40.799764 (+ 416us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:40.799770 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:40.799795 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:40.799909 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:40.800604 (+ 695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:40.800607 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:40.800608 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:40.800611 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:40.800648 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:40.800658 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:40.800714 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:40.800890 (+ 176us) server_negotiation.cc:299] Negotiation successful
0809 02:18:40.800925 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
I20260809 02:18:40.801488 31923 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:40.801858 31923 log.cc:824] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:40.802155 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:40.802719 31923 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1: No bootstrap required, opened a new log
I20260809 02:18:40.805090 31926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:40.803933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51383 (local address 127.0.0.1:45381)
0809 02:18:40.803973 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:40.803975 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:40.804091 (+ 116us) server_negotiation.cc:407] Connection header received
0809 02:18:40.804115 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:40.804116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:40.804121 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:40.804204 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:40.804844 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:40.804845 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:40.804845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:40.804851 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:40.804868 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:40.804874 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:40.804985 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:40.805003 (+ 18us) server_negotiation.cc:299] Negotiation successful
0809 02:18:40.805018 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:18:40.805348 31923 raft_consensus.cc:348] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260809 02:18:40.805505 31923 raft_consensus.cc:374] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:40.805547 31923 raft_consensus.cc:729] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b6fdcfd8431e4144aa342184d85d20a1, State: Initialized, Role: FOLLOWER
I20260809 02:18:40.805660 31923 consensus_queue.cc:260] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260809 02:18:40.805754 31923 raft_consensus.cc:388] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:40.805801 31923 raft_consensus.cc:482] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:40.805873 31923 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:40.806449 31923 raft_consensus.cc:504] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260809 02:18:40.806567 31923 leader_election.cc:302] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b6fdcfd8431e4144aa342184d85d20a1; no voters:
I20260809 02:18:40.806775 31923 leader_election.cc:288] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:40.806807 31933 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:40.806926 31933 raft_consensus.cc:686] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [term 1 LEADER]: Becoming Leader. State: Replica: b6fdcfd8431e4144aa342184d85d20a1, State: Running, Role: LEADER
I20260809 02:18:40.807006 31933 consensus_queue.cc:237] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } }
I20260809 02:18:40.807030 31923 sys_catalog.cc:564] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:40.807337 31934 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b6fdcfd8431e4144aa342184d85d20a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } } }
I20260809 02:18:40.807442 31934 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:40.807653 31935 sys_catalog.cc:455] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b6fdcfd8431e4144aa342184d85d20a1. Latest consensus state: current_term: 1 leader_uuid: "b6fdcfd8431e4144aa342184d85d20a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b6fdcfd8431e4144aa342184d85d20a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45381 } } }
I20260809 02:18:40.807726 31935 sys_catalog.cc:458] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:40.808112 31941 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:40.808570 31941 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:40.810159 31941 catalog_manager.cc:1347] Generated new cluster ID: 3b76b240f3a34740adee93a81a277e3b
I20260809 02:18:40.810212 31941 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:40.819841 31941 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:40.820007 31941 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:40.823912 31941 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b6fdcfd8431e4144aa342184d85d20a1: Generated new TSK 0
I20260809 02:18:40.824098 31941 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:40.826978 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31847
2026-08-09T02:18:40Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1382 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:40 dist-test-slave-qg60 krb5kdc[31956](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:40 dist-test-slave-qg60 krb5kdc[31956](info): set up 2 sockets
Aug 09 02:18:40 dist-test-slave-qg60 krb5kdc[31956](info): commencing operation
krb5kdc: starting...
W20260809 02:18:41.926354 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[31956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:42.056062 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36369
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37723
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36369
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:42.140439 31971 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:42.140741 31971 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:42.140836 31971 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:42.142063 31971 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:42.142143 31971 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:42.142169 31971 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:42.142197 31971 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:42.142225 31971 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:42.143771 31971 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37723
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36369
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36369
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31971
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:42.144078 31971 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:42.144323 31971 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:42.149500 31982 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:42.149518 31971 server_base.cc:1034] running on GCE node
W20260809 02:18:42.149501 31980 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:42.149518 31978 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:42.150022 31971 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:42.150394 31971 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:42.151561 31971 hybrid_clock.cc:648] HybridClock initialized: now 1786241922151530 us; error 45 us; skew 500 ppm
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[31956](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241922, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:42.154342 31971 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:42.154827 31971 webserver.cc:460] Webserver started at http://127.0.0.1:36867/ using document root <none> and password file <none>
I20260809 02:18:42.155027 31971 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:42.155068 31971 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:42.155164 31971 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:42.156225 31971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "349acf0a234847d4a96a8656a5e8aff4"
format_stamp: "Formatted at 2026-08-09 02:18:42 on dist-test-slave-qg60"
server_key: "727c9c056bddcb4b1fb52f65185aae47"
server_key_iv: "7975275e30d48089f957525e6c45c63d"
server_key_version: "encryptionkey@0"
I20260809 02:18:42.156541 31971 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "349acf0a234847d4a96a8656a5e8aff4"
format_stamp: "Formatted at 2026-08-09 02:18:42 on dist-test-slave-qg60"
server_key: "727c9c056bddcb4b1fb52f65185aae47"
server_key_iv: "7975275e30d48089f957525e6c45c63d"
server_key_version: "encryptionkey@0"
I20260809 02:18:42.157991 31971 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:18:42.158731 31989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:42.158960 31971 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:42.159065 31971 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "349acf0a234847d4a96a8656a5e8aff4"
format_stamp: "Formatted at 2026-08-09 02:18:42 on dist-test-slave-qg60"
server_key: "727c9c056bddcb4b1fb52f65185aae47"
server_key_iv: "7975275e30d48089f957525e6c45c63d"
server_key_version: "encryptionkey@0"
I20260809 02:18:42.159147 31971 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:42.174160 31971 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:42.176050 31971 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:42.176205 31971 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:42.180863 31971 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36369
I20260809 02:18:42.180918 32046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36369 every 8 connection(s)
I20260809 02:18:42.181298 31971 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:42.181571 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 31971
I20260809 02:18:42.181706 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:42.181932 9377 external_mini_cluster.cc:1428] Setting key 5856b62f41f7e161359f054f3270846d
I20260809 02:18:42.182554 32047 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:42.185763 32047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4: Bootstrap starting.
I20260809 02:18:42.186506 32047 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:42.186797 32047 log.cc:824] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:42.187577 32047 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4: No bootstrap required, opened a new log
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[31956](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:18:42.190035 32047 raft_consensus.cc:348] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 02:18:42.190205 32047 raft_consensus.cc:374] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:42.190256 32047 raft_consensus.cc:729] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 349acf0a234847d4a96a8656a5e8aff4, State: Initialized, Role: FOLLOWER
I20260809 02:18:42.190408 32047 consensus_queue.cc:260] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 02:18:42.190505 32047 raft_consensus.cc:388] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:42.190548 32047 raft_consensus.cc:482] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:42.190608 32047 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:42.191434 32047 raft_consensus.cc:504] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 02:18:42.191560 32047 leader_election.cc:302] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 349acf0a234847d4a96a8656a5e8aff4; no voters:
I20260809 02:18:42.191752 32047 leader_election.cc:288] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:42.191823 32053 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:42.192018 32053 raft_consensus.cc:686] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [term 1 LEADER]: Becoming Leader. State: Replica: 349acf0a234847d4a96a8656a5e8aff4, State: Running, Role: LEADER
I20260809 02:18:42.192050 32047 sys_catalog.cc:564] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:42.192121 32053 consensus_queue.cc:237] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } }
I20260809 02:18:42.192468 32053 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 349acf0a234847d4a96a8656a5e8aff4. Latest consensus state: current_term: 1 leader_uuid: "349acf0a234847d4a96a8656a5e8aff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } } }
I20260809 02:18:42.192479 32054 sys_catalog.cc:455] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "349acf0a234847d4a96a8656a5e8aff4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "349acf0a234847d4a96a8656a5e8aff4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36369 } } }
I20260809 02:18:42.192621 32054 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:42.192873 32053 sys_catalog.cc:458] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:42.193038 32060 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:42.194278 32060 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:42.194518 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:42.182465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38209 (local address 127.0.0.1:36369)
0809 02:18:42.182669 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:42.182677 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:42.182703 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:18:42.183156 (+ 453us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:42.183163 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:42.183215 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:42.183328 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:42.184155 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:42.185198 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:42.186285 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:42.186904 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:42.186996 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:42.189906 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:42.189910 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:42.189910 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:42.189922 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:42.192047 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:42.192507 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:42.192511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:42.192512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:42.192634 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:42.192905 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:42.192906 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:42.192907 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:42.193277 (+ 370us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:42.194108 (+ 831us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:42.194182 (+ 74us) server_negotiation.cc:299] Negotiation successful
0809 02:18:42.194295 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":108,"threads_started":1}
I20260809 02:18:42.195526 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:42.195878 32060 catalog_manager.cc:1347] Generated new cluster ID: fb134c658ed14172bcb910d9c0c254be
I20260809 02:18:42.195928 32060 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:42.204996 32050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:42.197129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56935 (local address 127.0.0.1:36369)
0809 02:18:42.197168 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:42.197170 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:42.197225 (+ 55us) server_negotiation.cc:407] Connection header received
0809 02:18:42.197306 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:42.197307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:42.197317 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:42.197377 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:42.198009 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:42.198773 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:42.199840 (+ 1067us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:42.200163 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:42.200208 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:42.201903 (+ 1695us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:42.201904 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:42.201905 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:42.201912 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:42.203664 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:42.204068 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:42.204070 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:42.204070 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:42.204138 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:42.204356 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:42.204357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:42.204358 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:42.204487 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:42.204560 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:42.204816 (+ 256us) server_negotiation.cc:299] Negotiation successful
0809 02:18:42.204895 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:18:42.209043 32060 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:42.209564 32060 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:42.214598 32060 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 349acf0a234847d4a96a8656a5e8aff4: Generated new TSK 0
I20260809 02:18:42.214779 32060 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:42.216763 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 31971
2026-08-09T02:18:42Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1390 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[32081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[32081](info): set up 2 sockets
Aug 09 02:18:42 dist-test-slave-qg60 krb5kdc[32081](info): commencing operation
krb5kdc: starting...
W20260809 02:18:43.381130 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:43 dist-test-slave-qg60 krb5kdc[32081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241923, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:43Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:43.510118 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44673
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37559
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44673
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:43.593410 32095 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:43.593633 32095 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:43.593657 32095 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:43.594849 32095 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:43.594902 32095 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:43.594916 32095 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:43.594930 32095 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:43.594985 32095 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:43.596570 32095 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37559
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44673
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44673
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32095
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:43.596875 32095 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:43.597143 32095 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:43.602197 32100 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:43.602345 32101 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:43.602334 32095 server_base.cc:1034] running on GCE node
W20260809 02:18:43.602276 32103 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:43.602761 32095 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:43.603086 32095 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:43.604233 32095 hybrid_clock.cc:648] HybridClock initialized: now 1786241923604209 us; error 34 us; skew 500 ppm
Aug 09 02:18:43 dist-test-slave-qg60 krb5kdc[32081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241923, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:43.611873 32095 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:43.612416 32095 webserver.cc:460] Webserver started at http://127.0.0.1:46341/ using document root <none> and password file <none>
I20260809 02:18:43.612682 32095 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:43.612772 32095 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:43.612931 32095 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:43.614024 32095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "a31e5ece8a504016926fdabb8d6feab0"
format_stamp: "Formatted at 2026-08-09 02:18:43 on dist-test-slave-qg60"
server_key: "94e708b4d5a5b81d5d7b6bac441e6aa2"
server_key_iv: "761c6676d7ca4524d3ef86e7c21fa92c"
server_key_version: "encryptionkey@0"
I20260809 02:18:43.614393 32095 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "a31e5ece8a504016926fdabb8d6feab0"
format_stamp: "Formatted at 2026-08-09 02:18:43 on dist-test-slave-qg60"
server_key: "94e708b4d5a5b81d5d7b6bac441e6aa2"
server_key_iv: "761c6676d7ca4524d3ef86e7c21fa92c"
server_key_version: "encryptionkey@0"
I20260809 02:18:43.615857 32095 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:18:43.616701 32110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:43.617075 32095 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:43.617203 32095 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "a31e5ece8a504016926fdabb8d6feab0"
format_stamp: "Formatted at 2026-08-09 02:18:43 on dist-test-slave-qg60"
server_key: "94e708b4d5a5b81d5d7b6bac441e6aa2"
server_key_iv: "761c6676d7ca4524d3ef86e7c21fa92c"
server_key_version: "encryptionkey@0"
I20260809 02:18:43.617292 32095 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:43.644251 32095 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:43.646710 32095 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:43.646890 32095 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:43.651458 32095 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44673
I20260809 02:18:43.651499 32169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44673 every 8 connection(s)
I20260809 02:18:43.651887 32095 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:43.652933 32170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:43.655287 32170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0: Bootstrap starting.
I20260809 02:18:43.655867 32170 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:43.656031 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32095
I20260809 02:18:43.656172 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:43.656224 32170 log.cc:824] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:43.656427 9377 external_mini_cluster.cc:1428] Setting key becd229eff8f9237775141866e344088
I20260809 02:18:43.657181 32170 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0: No bootstrap required, opened a new log
I20260809 02:18:43.659415 32170 raft_consensus.cc:348] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260809 02:18:43.659578 32170 raft_consensus.cc:374] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:43.659631 32170 raft_consensus.cc:729] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a31e5ece8a504016926fdabb8d6feab0, State: Initialized, Role: FOLLOWER
I20260809 02:18:43.659763 32170 consensus_queue.cc:260] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260809 02:18:43.659845 32170 raft_consensus.cc:388] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:43.659891 32170 raft_consensus.cc:482] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:43.659940 32170 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:43.660514 32170 raft_consensus.cc:504] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260809 02:18:43.660635 32170 leader_election.cc:302] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a31e5ece8a504016926fdabb8d6feab0; no voters:
I20260809 02:18:43.660890 32170 leader_election.cc:288] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:43.660931 32176 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:43.661087 32176 raft_consensus.cc:686] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [term 1 LEADER]: Becoming Leader. State: Replica: a31e5ece8a504016926fdabb8d6feab0, State: Running, Role: LEADER
I20260809 02:18:43.661237 32170 sys_catalog.cc:564] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:43.661221 32176 consensus_queue.cc:237] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } }
I20260809 02:18:43.661602 32178 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a31e5ece8a504016926fdabb8d6feab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } } }
I20260809 02:18:43.661691 32178 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:43.661890 32177 sys_catalog.cc:455] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a31e5ece8a504016926fdabb8d6feab0. Latest consensus state: current_term: 1 leader_uuid: "a31e5ece8a504016926fdabb8d6feab0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a31e5ece8a504016926fdabb8d6feab0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44673 } } }
I20260809 02:18:43.661948 32177 sys_catalog.cc:458] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:43.662369 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:43.657064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36107 (local address 127.0.0.1:44673)
0809 02:18:43.657268 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:43.657278 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:43.657313 (+ 35us) server_negotiation.cc:407] Connection header received
0809 02:18:43.657746 (+ 433us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:43.657753 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:43.657799 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:43.657951 (+ 152us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:43.658187 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:43.659090 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:43.660116 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:43.660545 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:43.660668 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:43.661436 (+ 768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:43.661440 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:43.661440 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:43.661661 (+ 221us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:43.661702 (+ 41us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:43.661718 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:43.661786 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:43.662084 (+ 298us) server_negotiation.cc:299] Negotiation successful
0809 02:18:43.662155 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143,"thread_start_us":90,"threads_started":1}
I20260809 02:18:43.662442 32184 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:43.662904 32184 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:43.663452 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:43.664582 32184 catalog_manager.cc:1347] Generated new cluster ID: 55cb2bbc56ea4b8b893960c95753685f
I20260809 02:18:43.664685 32184 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:43.671729 32184 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:43.672283 32175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:43.669225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40491 (local address 127.0.0.1:44673)
0809 02:18:43.669276 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:43.669278 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:43.669383 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:18:43.669447 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:43.669448 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:43.669458 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:43.669540 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:43.669679 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:43.670163 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:43.670918 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:43.671137 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:43.671219 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:43.672013 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:43.672015 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:43.672015 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:43.672020 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:43.672035 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:43.672040 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:43.672093 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:43.672157 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:18:43.672179 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":11,"server-negotiator.queue_time_us":25}
I20260809 02:18:43.672444 32184 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:43.685698 32184 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a31e5ece8a504016926fdabb8d6feab0: Generated new TSK 0
I20260809 02:18:43.685925 32184 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:43.692366 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32095
2026-08-09T02:18:43Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1473 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:43 dist-test-slave-qg60 krb5kdc[32206](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:43 dist-test-slave-qg60 krb5kdc[32206](info): set up 2 sockets
Aug 09 02:18:43 dist-test-slave-qg60 krb5kdc[32206](info): commencing operation
krb5kdc: starting...
W20260809 02:18:44.792374 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:44 dist-test-slave-qg60 krb5kdc[32206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:44.920914 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38931
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38931
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:45.004009 32220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:45.004235 32220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:45.004259 32220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:45.005470 32220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:45.005517 32220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:45.005532 32220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:45.005544 32220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:45.005556 32220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:45.007054 32220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:45.007261 32220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:45.007441 32220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:45.012400 32225 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:45.012406 32226 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:45.012413 32228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:45.012696 32220 server_base.cc:1034] running on GCE node
I20260809 02:18:45.013023 32220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:45.013335 32220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:45.014501 32220 hybrid_clock.cc:648] HybridClock initialized: now 1786241925014479 us; error 49 us; skew 500 ppm
Aug 09 02:18:45 dist-test-slave-qg60 krb5kdc[32206](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241925, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:45.017138 32220 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:45.017645 32220 webserver.cc:460] Webserver started at http://127.0.0.1:40761/ using document root <none> and password file <none>
I20260809 02:18:45.017913 32220 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:45.017963 32220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:45.018150 32220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:45.019243 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "a2f19d1362744bad994ff9a68d290128"
format_stamp: "Formatted at 2026-08-09 02:18:45 on dist-test-slave-qg60"
server_key: "f555792f38923ea215b0f159b042b55d"
server_key_iv: "b236e7da01447da31b7f0bef745fdb05"
server_key_version: "encryptionkey@0"
I20260809 02:18:45.019608 32220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "a2f19d1362744bad994ff9a68d290128"
format_stamp: "Formatted at 2026-08-09 02:18:45 on dist-test-slave-qg60"
server_key: "f555792f38923ea215b0f159b042b55d"
server_key_iv: "b236e7da01447da31b7f0bef745fdb05"
server_key_version: "encryptionkey@0"
I20260809 02:18:45.021087 32220 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:18:45.021950 32235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:45.022186 32220 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:45.022302 32220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "a2f19d1362744bad994ff9a68d290128"
format_stamp: "Formatted at 2026-08-09 02:18:45 on dist-test-slave-qg60"
server_key: "f555792f38923ea215b0f159b042b55d"
server_key_iv: "b236e7da01447da31b7f0bef745fdb05"
server_key_version: "encryptionkey@0"
I20260809 02:18:45.022382 32220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:45.025925 32220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:45.028141 32220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:45.028306 32220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:45.032800 32220 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38931
I20260809 02:18:45.032835 32292 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38931 every 8 connection(s)
I20260809 02:18:45.033191 32220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:45.034219 32293 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:45.036545 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32220
I20260809 02:18:45.036561 32293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128: Bootstrap starting.
I20260809 02:18:45.036696 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:45.036919 9377 external_mini_cluster.cc:1428] Setting key df7f530512b814883f9adb739a689f77
I20260809 02:18:45.037382 32293 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:45.037779 32293 log.cc:824] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:45.038617 32293 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128: No bootstrap required, opened a new log
I20260809 02:18:45.039760 32296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:45.037456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48035 (local address 127.0.0.1:38931)
0809 02:18:45.037704 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:45.037715 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:45.037757 (+ 42us) server_negotiation.cc:407] Connection header received
0809 02:18:45.038376 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:45.038385 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:45.038420 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:45.038576 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:45.039314 (+ 738us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:45.039318 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:45.039319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:45.039323 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:45.039369 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:45.039383 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:45.039486 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:45.039517 (+ 31us) server_negotiation.cc:299] Negotiation successful
0809 02:18:45.039557 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260809 02:18:45.040916 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:45.041177 32293 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } }
I20260809 02:18:45.041339 32293 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:45.041379 32293 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2f19d1362744bad994ff9a68d290128, State: Initialized, Role: FOLLOWER
I20260809 02:18:45.041514 32293 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } }
I20260809 02:18:45.041615 32293 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:45.041647 32293 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:45.041704 32293 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:45.042469 32293 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } }
I20260809 02:18:45.042609 32293 leader_election.cc:302] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2f19d1362744bad994ff9a68d290128; no voters:
I20260809 02:18:45.042770 32293 leader_election.cc:288] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:45.042815 32305 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:45.043035 32305 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [term 1 LEADER]: Becoming Leader. State: Replica: a2f19d1362744bad994ff9a68d290128, State: Running, Role: LEADER
I20260809 02:18:45.043074 32293 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:45.043561 32305 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } }
I20260809 02:18:45.043789 32296 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:45.042554 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32971 (local address 127.0.0.1:38931)
0809 02:18:45.042595 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:45.042597 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:45.042753 (+ 156us) server_negotiation.cc:407] Connection header received
0809 02:18:45.042818 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:45.042818 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:45.042824 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:45.042888 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:45.043533 (+ 645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:45.043534 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:45.043535 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:45.043539 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:45.043553 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:45.043558 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:45.043614 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:45.043719 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:18:45.043735 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:18:45.044189 32307 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2f19d1362744bad994ff9a68d290128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } } }
I20260809 02:18:45.044292 32307 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:45.044610 32316 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:45.044595 32308 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2f19d1362744bad994ff9a68d290128. Latest consensus state: current_term: 1 leader_uuid: "a2f19d1362744bad994ff9a68d290128" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2f19d1362744bad994ff9a68d290128" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38931 } } }
I20260809 02:18:45.044762 32308 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:45.045197 32316 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:45.046761 32316 catalog_manager.cc:1347] Generated new cluster ID: d5869113067f49a88cf08fcea44b2e00
I20260809 02:18:45.046885 32316 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:45.054199 32316 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:45.054405 32316 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:45.056486 32316 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2f19d1362744bad994ff9a68d290128: Generated new TSK 0
I20260809 02:18:45.056682 32316 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:45.062636 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32220
2026-08-09T02:18:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1369 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:45 dist-test-slave-qg60 krb5kdc[32330](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:45 dist-test-slave-qg60 krb5kdc[32330](info): set up 2 sockets
Aug 09 02:18:45 dist-test-slave-qg60 krb5kdc[32330](info): commencing operation
krb5kdc: starting...
W20260809 02:18:46.179031 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:46.311375 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38165
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40587
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38165
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:46.397164 32345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:46.397413 32345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:46.397450 32345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:46.398638 32345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:46.398705 32345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:46.398729 32345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:46.398751 32345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:46.398778 32345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:46.400355 32345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40587
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38165
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38165
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:46.400669 32345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:46.400954 32345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:46.406075 32345 server_base.cc:1034] running on GCE node
W20260809 02:18:46.406019 32350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:46.406044 32351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:46.406044 32353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:46.406576 32345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:46.406839 32345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:46.407980 32345 hybrid_clock.cc:648] HybridClock initialized: now 1786241926407967 us; error 32 us; skew 500 ppm
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32330](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241926, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:46.410629 32345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:46.411074 32345 webserver.cc:460] Webserver started at http://127.0.0.1:33439/ using document root <none> and password file <none>
I20260809 02:18:46.411393 32345 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:46.411463 32345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:46.411609 32345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:46.412680 32345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "a4273c718ede4293a4c587efac924345"
format_stamp: "Formatted at 2026-08-09 02:18:46 on dist-test-slave-qg60"
server_key: "0f8eac28b56e9f1fa94ed7db268f317f"
server_key_iv: "190a1a4a1324537288959d0a3758158c"
server_key_version: "encryptionkey@0"
I20260809 02:18:46.413111 32345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "a4273c718ede4293a4c587efac924345"
format_stamp: "Formatted at 2026-08-09 02:18:46 on dist-test-slave-qg60"
server_key: "0f8eac28b56e9f1fa94ed7db268f317f"
server_key_iv: "190a1a4a1324537288959d0a3758158c"
server_key_version: "encryptionkey@0"
I20260809 02:18:46.414542 32345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:46.415306 32361 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:46.415520 32345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:46.415612 32345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "a4273c718ede4293a4c587efac924345"
format_stamp: "Formatted at 2026-08-09 02:18:46 on dist-test-slave-qg60"
server_key: "0f8eac28b56e9f1fa94ed7db268f317f"
server_key_iv: "190a1a4a1324537288959d0a3758158c"
server_key_version: "encryptionkey@0"
I20260809 02:18:46.415693 32345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:46.427560 32345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:46.429562 32345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:46.429728 32345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:46.433874 32345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38165
I20260809 02:18:46.433909 32419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38165 every 8 connection(s)
I20260809 02:18:46.434315 32345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:46.435346 32420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:46.436834 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32345
I20260809 02:18:46.436967 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:46.437179 9377 external_mini_cluster.cc:1428] Setting key 25a486029f44b5358364fdf10ca51b55
I20260809 02:18:46.437994 32420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: Bootstrap starting.
I20260809 02:18:46.438591 32420 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:46.438903 32420 log.cc:824] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:46.439585 32420 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: No bootstrap required, opened a new log
I20260809 02:18:46.441399 32420 raft_consensus.cc:348] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } }
I20260809 02:18:46.441551 32420 raft_consensus.cc:374] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:46.441648 32420 raft_consensus.cc:729] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a4273c718ede4293a4c587efac924345, State: Initialized, Role: FOLLOWER
I20260809 02:18:46.441808 32420 consensus_queue.cc:260] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } }
I20260809 02:18:46.441913 32420 raft_consensus.cc:388] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:46.441949 32420 raft_consensus.cc:482] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:46.441995 32420 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:46.442587 32420 raft_consensus.cc:504] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } }
I20260809 02:18:46.442708 32420 leader_election.cc:302] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a4273c718ede4293a4c587efac924345; no voters:
I20260809 02:18:46.442911 32420 leader_election.cc:288] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:46.442938 32425 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:46.443063 32425 raft_consensus.cc:686] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [term 1 LEADER]: Becoming Leader. State: Replica: a4273c718ede4293a4c587efac924345, State: Running, Role: LEADER
I20260809 02:18:46.443203 32420 sys_catalog.cc:564] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:46.443203 32425 consensus_queue.cc:237] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } }
I20260809 02:18:46.443651 32425 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a4273c718ede4293a4c587efac924345. Latest consensus state: current_term: 1 leader_uuid: "a4273c718ede4293a4c587efac924345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } } }
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32330](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:18:46.443665 32426 sys_catalog.cc:455] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a4273c718ede4293a4c587efac924345" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a4273c718ede4293a4c587efac924345" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38165 } } }
I20260809 02:18:46.443848 32425 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:46.444113 32426 sys_catalog.cc:458] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345 [sys.catalog]: This master's current role is: LEADER
W20260809 02:18:46.445495 32443 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:46.445992 32443 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:46.446079 32434 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:46.446291 32434 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:46.447916 32434 catalog_manager.cc:1347] Generated new cluster ID: a919f1ab9d554587bbcd6cbbfacb26fa
I20260809 02:18:46.447971 32434 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:46.448882 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:46.437745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:35019 (local address 127.0.0.1:38165)
0809 02:18:46.437938 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:46.437947 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:46.437972 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:18:46.438465 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:46.438472 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:46.438520 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:46.438664 (+ 144us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:46.439331 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:46.440279 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:46.441132 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:46.441530 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:46.441649 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:46.444888 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:46.444892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:46.444893 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:46.444905 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:46.447063 (+ 2158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:46.447567 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:46.447570 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:46.447571 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:46.447674 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:46.447980 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:46.447981 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:46.447982 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:46.448256 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:46.448422 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:46.448553 (+ 131us) server_negotiation.cc:299] Negotiation successful
0809 02:18:46.448665 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":137,"thread_start_us":85,"threads_started":1}
I20260809 02:18:46.453842 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:46.457216 32434 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:46.457901 32434 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:46.465979 32434 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a4273c718ede4293a4c587efac924345: Generated new TSK 0
I20260809 02:18:46.466213 32434 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:46.467308 32423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:46.457005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:45543 (local address 127.0.0.1:38165)
0809 02:18:46.457041 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:46.457043 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:46.460865 (+ 3822us) server_negotiation.cc:407] Connection header received
0809 02:18:46.460950 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:46.460950 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:46.460961 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:46.461026 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:46.461621 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:46.462152 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:46.462890 (+ 738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:46.463119 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:46.463200 (+ 81us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:46.464454 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:46.464456 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:46.464456 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:46.464464 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:46.466077 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:46.466522 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:46.466524 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:46.466524 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:46.466594 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:46.466791 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:46.466792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:46.466793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:46.466920 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:46.467021 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:46.467141 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:18:46.467214 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:18:46.469995 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32345
W20260809 02:18:46.477392 32328 connection.cc:535] client connection to 127.0.0.1:38165 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38165): Connection reset by peer (error 104)
2026-08-09T02:18:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1407 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32456](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32456](info): set up 2 sockets
Aug 09 02:18:46 dist-test-slave-qg60 krb5kdc[32456](info): commencing operation
krb5kdc: starting...
W20260809 02:18:47.557267 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.066s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:47 dist-test-slave-qg60 krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241927, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:47.687584 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44099
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44099
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:47.776324 32469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:47.776558 32469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:47.776597 32469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:47.777740 32469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:47.777803 32469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:47.777827 32469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:47.777854 32469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:47.777884 32469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:47.779371 32469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44099
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44099
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:47.779649 32469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:47.779872 32469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:47.785115 32469 server_base.cc:1034] running on GCE node
W20260809 02:18:47.785065 32476 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:47.785073 32478 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:47.785065 32474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:47.785552 32469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:47.785880 32469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:47.787045 32469 hybrid_clock.cc:648] HybridClock initialized: now 1786241927787035 us; error 31 us; skew 500 ppm
Aug 09 02:18:47 dist-test-slave-qg60 krb5kdc[32456](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241927, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:47.789853 32469 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:47.790333 32469 webserver.cc:460] Webserver started at http://127.0.0.1:39007/ using document root <none> and password file <none>
I20260809 02:18:47.790584 32469 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:47.790637 32469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:47.790803 32469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:47.791891 32469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "a503d9e8cd3d42c69398287df910ed80"
format_stamp: "Formatted at 2026-08-09 02:18:47 on dist-test-slave-qg60"
server_key: "6ac2a5ad1b9b0b4e89a3659d09f465a2"
server_key_iv: "3820297853b2b5ad657ea26ee65461d9"
server_key_version: "encryptionkey@0"
I20260809 02:18:47.792259 32469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "a503d9e8cd3d42c69398287df910ed80"
format_stamp: "Formatted at 2026-08-09 02:18:47 on dist-test-slave-qg60"
server_key: "6ac2a5ad1b9b0b4e89a3659d09f465a2"
server_key_iv: "3820297853b2b5ad657ea26ee65461d9"
server_key_version: "encryptionkey@0"
I20260809 02:18:47.793831 32469 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:47.794647 32485 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:47.794836 32469 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:47.794937 32469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "a503d9e8cd3d42c69398287df910ed80"
format_stamp: "Formatted at 2026-08-09 02:18:47 on dist-test-slave-qg60"
server_key: "6ac2a5ad1b9b0b4e89a3659d09f465a2"
server_key_iv: "3820297853b2b5ad657ea26ee65461d9"
server_key_version: "encryptionkey@0"
I20260809 02:18:47.795011 32469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:47.819205 32469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:47.821981 32469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:47.822175 32469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:47.827188 32469 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44099
I20260809 02:18:47.827704 32469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:47.828055 32539 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44099 every 8 connection(s)
I20260809 02:18:47.828941 32540 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:47.831667 32540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80: Bootstrap starting.
I20260809 02:18:47.832295 32540 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:47.832598 32540 log.cc:824] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:47.833412 32540 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80: No bootstrap required, opened a new log
I20260809 02:18:47.833847 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32469
I20260809 02:18:47.833982 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:47.834244 9377 external_mini_cluster.cc:1428] Setting key 40e88f8731b12164a3894fb723de4f88
I20260809 02:18:47.835925 32540 raft_consensus.cc:348] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260809 02:18:47.836145 32540 raft_consensus.cc:374] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:47.836184 32540 raft_consensus.cc:729] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a503d9e8cd3d42c69398287df910ed80, State: Initialized, Role: FOLLOWER
I20260809 02:18:47.836330 32540 consensus_queue.cc:260] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260809 02:18:47.836422 32540 raft_consensus.cc:388] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:47.836458 32540 raft_consensus.cc:482] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:47.836519 32540 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:47.837330 32540 raft_consensus.cc:504] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260809 02:18:47.837441 32540 leader_election.cc:302] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a503d9e8cd3d42c69398287df910ed80; no voters:
I20260809 02:18:47.837615 32540 leader_election.cc:288] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:47.837692 32546 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:47.837863 32540 sys_catalog.cc:564] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:47.837899 32546 raft_consensus.cc:686] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [term 1 LEADER]: Becoming Leader. State: Replica: a503d9e8cd3d42c69398287df910ed80, State: Running, Role: LEADER
I20260809 02:18:47.838006 32546 consensus_queue.cc:237] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } }
I20260809 02:18:47.838387 32548 sys_catalog.cc:455] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a503d9e8cd3d42c69398287df910ed80. Latest consensus state: current_term: 1 leader_uuid: "a503d9e8cd3d42c69398287df910ed80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } } }
I20260809 02:18:47.838486 32548 sys_catalog.cc:458] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:47.838753 32550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:47.838878 32547 sys_catalog.cc:455] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a503d9e8cd3d42c69398287df910ed80" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a503d9e8cd3d42c69398287df910ed80" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44099 } } }
I20260809 02:18:47.839031 32547 sys_catalog.cc:458] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:47.840965 32550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:47.842350 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:47.834962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:45525 (local address 127.0.0.1:44099)
0809 02:18:47.835238 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:47.835247 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:47.835286 (+ 39us) server_negotiation.cc:407] Connection header received
0809 02:18:47.836159 (+ 873us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:47.836169 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:47.836237 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:47.836390 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:47.836749 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:47.838159 (+ 1410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:47.839414 (+ 1255us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:47.840426 (+ 1012us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:47.840548 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:47.841464 (+ 916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:47.841467 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:47.841468 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:47.841693 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:47.841732 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:47.841747 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:47.842045 (+ 298us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:47.842092 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:18:47.842135 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":168,"threads_started":1}
I20260809 02:18:47.843145 32550 catalog_manager.cc:1347] Generated new cluster ID: c0398daaafe7470194937ac40394369d
I20260809 02:18:47.843195 32550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:47.843569 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:47.848253 32545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:47.845072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:55345 (local address 127.0.0.1:44099)
0809 02:18:47.845118 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:47.845119 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:47.845251 (+ 132us) server_negotiation.cc:407] Connection header received
0809 02:18:47.845341 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:47.845342 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:47.845354 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:47.845404 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:47.845572 (+ 168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:47.846094 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:47.846895 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:47.847140 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:47.847232 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:47.847968 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:47.847970 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:47.847970 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:47.847975 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:47.847989 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:47.847994 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:47.848046 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:47.848134 (+ 88us) server_negotiation.cc:299] Negotiation successful
0809 02:18:47.848144 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:18:47.854506 32550 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:47.855093 32550 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:47.860210 32550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a503d9e8cd3d42c69398287df910ed80: Generated new TSK 0
I20260809 02:18:47.860410 32550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:47.864133 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32469
W20260809 02:18:47.873495 32454 connection.cc:535] client connection to 127.0.0.1:44099 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44099): Connection reset by peer (error 104)
2026-08-09T02:18:47Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1396 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:47 dist-test-slave-qg60 krb5kdc[32576](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:47 dist-test-slave-qg60 krb5kdc[32576](info): set up 2 sockets
Aug 09 02:18:47 dist-test-slave-qg60 krb5kdc[32576](info): commencing operation
krb5kdc: starting...
W20260809 02:18:48.958669 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.071s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:49 dist-test-slave-qg60 krb5kdc[32576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241929, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:49.089638 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38633
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40143
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38633
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:49.172422 32589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:49.172663 32589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:49.172688 32589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:49.173859 32589 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:49.173905 32589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:49.173920 32589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:49.173933 32589 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:49.173945 32589 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:49.175377 32589 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40143
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38633
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38633
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:49.175585 32589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:49.175765 32589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:49.180842 32589 server_base.cc:1034] running on GCE node
W20260809 02:18:49.180862 32594 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:49.180980 32595 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:49.180852 32597 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:49.181293 32589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:49.181587 32589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:49.182760 32589 hybrid_clock.cc:648] HybridClock initialized: now 1786241929182747 us; error 29 us; skew 500 ppm
Aug 09 02:18:49 dist-test-slave-qg60 krb5kdc[32576](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241929, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:49.190326 32589 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:49.190812 32589 webserver.cc:460] Webserver started at http://127.0.0.1:45345/ using document root <none> and password file <none>
I20260809 02:18:49.191062 32589 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:49.191128 32589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:49.191273 32589 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:49.192346 32589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "c1d0783fca80493a8a2866815574890f"
format_stamp: "Formatted at 2026-08-09 02:18:49 on dist-test-slave-qg60"
server_key: "ef908b246c7048d4ee0b77652ffee511"
server_key_iv: "bf2abac2a989462d0636617ba7374ba3"
server_key_version: "encryptionkey@0"
I20260809 02:18:49.192764 32589 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "c1d0783fca80493a8a2866815574890f"
format_stamp: "Formatted at 2026-08-09 02:18:49 on dist-test-slave-qg60"
server_key: "ef908b246c7048d4ee0b77652ffee511"
server_key_iv: "bf2abac2a989462d0636617ba7374ba3"
server_key_version: "encryptionkey@0"
I20260809 02:18:49.194195 32589 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:49.195021 32604 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:49.195255 32589 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:49.195358 32589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "c1d0783fca80493a8a2866815574890f"
format_stamp: "Formatted at 2026-08-09 02:18:49 on dist-test-slave-qg60"
server_key: "ef908b246c7048d4ee0b77652ffee511"
server_key_iv: "bf2abac2a989462d0636617ba7374ba3"
server_key_version: "encryptionkey@0"
I20260809 02:18:49.195461 32589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:49.199018 32589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:49.201308 32589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:49.201475 32589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:49.205781 32589 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38633
I20260809 02:18:49.205819 32661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38633 every 8 connection(s)
I20260809 02:18:49.206172 32589 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:49.207311 32662 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:49.209918 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f: Bootstrap starting.
I20260809 02:18:49.210587 32662 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:49.210908 32662 log.cc:824] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:49.211674 32662 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f: No bootstrap required, opened a new log
I20260809 02:18:49.213649 32662 raft_consensus.cc:348] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260809 02:18:49.213804 32662 raft_consensus.cc:374] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:49.213862 32662 raft_consensus.cc:729] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1d0783fca80493a8a2866815574890f, State: Initialized, Role: FOLLOWER
I20260809 02:18:49.214051 32662 consensus_queue.cc:260] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260809 02:18:49.214124 32662 raft_consensus.cc:388] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:49.214149 32662 raft_consensus.cc:482] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:49.214221 32662 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:49.214825 32662 raft_consensus.cc:504] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260809 02:18:49.214946 32662 leader_election.cc:302] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c1d0783fca80493a8a2866815574890f; no voters:
I20260809 02:18:49.215157 32662 leader_election.cc:288] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:49.215199 32665 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:49.215415 32665 raft_consensus.cc:686] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [term 1 LEADER]: Becoming Leader. State: Replica: c1d0783fca80493a8a2866815574890f, State: Running, Role: LEADER
I20260809 02:18:49.215413 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32589
I20260809 02:18:49.215451 32662 sys_catalog.cc:564] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:49.215531 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:49.215586 32665 consensus_queue.cc:237] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } }
I20260809 02:18:49.215812 9377 external_mini_cluster.cc:1428] Setting key c5baa10e465a62fec4215d4f05d4cf3b
I20260809 02:18:49.216385 32665 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c1d0783fca80493a8a2866815574890f. Latest consensus state: current_term: 1 leader_uuid: "c1d0783fca80493a8a2866815574890f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } } }
I20260809 02:18:49.216823 32665 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:49.216707 32666 sys_catalog.cc:455] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c1d0783fca80493a8a2866815574890f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c1d0783fca80493a8a2866815574890f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38633 } } }
I20260809 02:18:49.217043 32666 sys_catalog.cc:458] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:49.217073 32680 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:49.217893 32680 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:49.219247 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:49.216395 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:51011 (local address 127.0.0.1:38633)
0809 02:18:49.216643 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:49.216659 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:49.217280 (+ 621us) server_negotiation.cc:407] Connection header received
0809 02:18:49.217512 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:49.217520 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:49.217552 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:49.217682 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:49.218749 (+ 1067us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:49.218753 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:49.218753 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:49.218758 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:49.218811 (+ 53us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:49.218827 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:49.218908 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:49.218989 (+ 81us) server_negotiation.cc:299] Negotiation successful
0809 02:18:49.219035 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":96,"threads_started":1}
I20260809 02:18:49.219489 32680 catalog_manager.cc:1347] Generated new cluster ID: 07df70168aa94b2aba9df4186c4a1f89
I20260809 02:18:49.219542 32680 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:49.220256 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:49.226023 32676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:49.224869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:45239 (local address 127.0.0.1:38633)
0809 02:18:49.224923 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:49.224925 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:49.224938 (+ 13us) server_negotiation.cc:407] Connection header received
0809 02:18:49.224969 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:49.224970 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:49.224979 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:49.225097 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:49.225761 (+ 664us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:49.225763 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:49.225763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:49.225768 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:49.225788 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:49.225793 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:49.225862 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:49.225905 (+ 43us) server_negotiation.cc:299] Negotiation successful
0809 02:18:49.225919 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:18:49.229391 32680 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:49.229557 32680 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:49.242143 32680 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c1d0783fca80493a8a2866815574890f: Generated new TSK 0
I20260809 02:18:49.242360 32680 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:49.247179 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32589
2026-08-09T02:18:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:49 dist-test-slave-qg60 krb5kdc[32695](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:49 dist-test-slave-qg60 krb5kdc[32695](info): set up 2 sockets
Aug 09 02:18:49 dist-test-slave-qg60 krb5kdc[32695](info): commencing operation
krb5kdc: starting...
W20260809 02:18:50.338809 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[32695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:50.471639 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44045
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:43589
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44045
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:50.556262 32715 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:50.556529 32715 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:50.556571 32715 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:50.557785 32715 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:50.557858 32715 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:50.557881 32715 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:50.557904 32715 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:50.557935 32715 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:50.559825 32715 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:43589
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44045
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44045
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32715
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:50.560146 32715 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:50.560392 32715 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:50.565450 32720 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:50.565485 32724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:50.565490 32722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:50.565707 32715 server_base.cc:1034] running on GCE node
I20260809 02:18:50.565976 32715 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:50.566299 32715 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:50.567442 32715 hybrid_clock.cc:648] HybridClock initialized: now 1786241930567430 us; error 32 us; skew 500 ppm
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[32695](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241930, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:50.570235 32715 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:50.570677 32715 webserver.cc:460] Webserver started at http://127.0.0.1:42483/ using document root <none> and password file <none>
I20260809 02:18:50.570923 32715 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:50.570967 32715 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:50.571132 32715 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:50.572290 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "2ecba9abd4f541a7a92ba4661b20611b"
format_stamp: "Formatted at 2026-08-09 02:18:50 on dist-test-slave-qg60"
server_key: "a43861e3ee356178f7835508a0e4637c"
server_key_iv: "84a646d5352869caeda1a06b3c00392a"
server_key_version: "encryptionkey@0"
I20260809 02:18:50.572772 32715 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "2ecba9abd4f541a7a92ba4661b20611b"
format_stamp: "Formatted at 2026-08-09 02:18:50 on dist-test-slave-qg60"
server_key: "a43861e3ee356178f7835508a0e4637c"
server_key_iv: "84a646d5352869caeda1a06b3c00392a"
server_key_version: "encryptionkey@0"
I20260809 02:18:50.574296 32715 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:50.575085 32732 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:50.575265 32715 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:50.575354 32715 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "2ecba9abd4f541a7a92ba4661b20611b"
format_stamp: "Formatted at 2026-08-09 02:18:50 on dist-test-slave-qg60"
server_key: "a43861e3ee356178f7835508a0e4637c"
server_key_iv: "84a646d5352869caeda1a06b3c00392a"
server_key_version: "encryptionkey@0"
I20260809 02:18:50.575429 32715 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:50.591681 32715 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:50.593732 32715 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:50.593890 32715 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:50.598197 32715 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44045
I20260809 02:18:50.598258 320 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44045 every 8 connection(s)
I20260809 02:18:50.598628 32715 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:50.599682 321 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:50.602154 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: Bootstrap starting.
I20260809 02:18:50.602756 321 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:50.603058 321 log.cc:824] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:50.603760 321 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: No bootstrap required, opened a new log
I20260809 02:18:50.605537 321 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260809 02:18:50.605677 321 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:50.605722 321 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ecba9abd4f541a7a92ba4661b20611b, State: Initialized, Role: FOLLOWER
I20260809 02:18:50.605878 321 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260809 02:18:50.605942 321 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:50.605964 321 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:50.606007 321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:50.606559 321 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260809 02:18:50.606648 321 leader_election.cc:302] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2ecba9abd4f541a7a92ba4661b20611b; no voters:
I20260809 02:18:50.606832 321 leader_election.cc:288] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:50.606873 325 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:50.607041 325 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [term 1 LEADER]: Becoming Leader. State: Replica: 2ecba9abd4f541a7a92ba4661b20611b, State: Running, Role: LEADER
I20260809 02:18:50.607105 321 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:50.607183 325 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } }
I20260809 02:18:50.607659 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 32715
I20260809 02:18:50.607802 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:50.608093 9377 external_mini_cluster.cc:1428] Setting key 8e124bc9c41f4b52dda97f228ace4956
I20260809 02:18:50.608397 327 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2ecba9abd4f541a7a92ba4661b20611b. Latest consensus state: current_term: 1 leader_uuid: "2ecba9abd4f541a7a92ba4661b20611b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } } }
I20260809 02:18:50.608506 327 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:50.608741 326 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2ecba9abd4f541a7a92ba4661b20611b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2ecba9abd4f541a7a92ba4661b20611b" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44045 } } }
I20260809 02:18:50.609061 326 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b [sys.catalog]: This master's current role is: LEADER
W20260809 02:18:50.609217 344 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:50.609265 344 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:50.609310 340 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:50.609462 340 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:50.611071 340 catalog_manager.cc:1347] Generated new cluster ID: d9b2a6f10e9b4f66b1fff9ee08e484e4
I20260809 02:18:50.611130 340 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[32695](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241930, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260809 02:18:50.621553 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:50.608893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:49969 (local address 127.0.0.1:44045)
0809 02:18:50.609585 (+ 692us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:50.609597 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:50.609634 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:18:50.609745 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:50.609751 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:50.609801 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:50.609943 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:50.610806 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:50.612188 (+ 1382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:50.613400 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:50.614047 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:50.614168 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:50.617434 (+ 3266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:50.617439 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:50.617440 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:50.617454 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:50.619627 (+ 2173us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:50.620122 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:50.620126 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:50.620126 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:50.620247 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:50.620497 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:50.620498 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:50.620498 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:50.620877 (+ 379us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:50.621075 (+ 198us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:50.621190 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:18:50.621297 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":628,"thread_start_us":90,"threads_started":1}
I20260809 02:18:50.622537 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:50.632444 345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:50.624144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:50225 (local address 127.0.0.1:44045)
0809 02:18:50.624182 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:50.624184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:50.624434 (+ 250us) server_negotiation.cc:407] Connection header received
0809 02:18:50.624524 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:50.624525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:50.624536 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:50.624567 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:50.625349 (+ 782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:50.626193 (+ 844us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:50.627304 (+ 1111us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:50.627651 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:50.627693 (+ 42us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:50.629374 (+ 1681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:50.629376 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:50.629376 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:50.629384 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:50.631049 (+ 1665us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:50.631521 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:50.631522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:50.631522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:50.631582 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:50.631813 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:50.631814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:50.631814 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:50.631941 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:50.632015 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:50.632268 (+ 253us) server_negotiation.cc:299] Negotiation successful
0809 02:18:50.632338 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:18:50.636888 340 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:50.637480 340 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:50.641076 340 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2ecba9abd4f541a7a92ba4661b20611b: Generated new TSK 0
I20260809 02:18:50.641268 340 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:50.646181 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 32715
W20260809 02:18:50.654851 32693 connection.cc:535] client connection to 127.0.0.1:44045 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44045): Connection reset by peer (error 104)
2026-08-09T02:18:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[355](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[355](info): set up 2 sockets
Aug 09 02:18:50 dist-test-slave-qg60 krb5kdc[355](info): commencing operation
krb5kdc: starting...
W20260809 02:18:51.737246 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:51 dist-test-slave-qg60 krb5kdc[355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241931, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:51.867719 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40079
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34305
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40079
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:51.951553 368 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:51.951766 368 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:51.951789 368 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:51.952982 368 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:51.953024 368 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:51.953039 368 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:51.953052 368 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:51.953064 368 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:51.954517 368 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34305
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40079
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40079
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.368
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:51.954716 368 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:51.954883 368 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:51.959770 373 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:51.959914 368 server_base.cc:1034] running on GCE node
W20260809 02:18:51.959779 374 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:51.959779 376 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:51.960345 368 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:51.960647 368 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:51.961810 368 hybrid_clock.cc:648] HybridClock initialized: now 1786241931961794 us; error 40 us; skew 500 ppm
Aug 09 02:18:51 dist-test-slave-qg60 krb5kdc[355](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241931, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:51.964350 368 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:51.964841 368 webserver.cc:460] Webserver started at http://127.0.0.1:43505/ using document root <none> and password file <none>
I20260809 02:18:51.965072 368 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:51.965137 368 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:51.965272 368 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:51.966347 368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "d4f00e69d1a7436589af0eaf8bbc52cd"
format_stamp: "Formatted at 2026-08-09 02:18:51 on dist-test-slave-qg60"
server_key: "b9f4194ab568d115caa605f83cca8d50"
server_key_iv: "188267f1e4aaa555b7e9564528970e98"
server_key_version: "encryptionkey@0"
I20260809 02:18:51.966713 368 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "d4f00e69d1a7436589af0eaf8bbc52cd"
format_stamp: "Formatted at 2026-08-09 02:18:51 on dist-test-slave-qg60"
server_key: "b9f4194ab568d115caa605f83cca8d50"
server_key_iv: "188267f1e4aaa555b7e9564528970e98"
server_key_version: "encryptionkey@0"
I20260809 02:18:51.968065 368 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:51.968951 385 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:51.969141 368 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:51.969239 368 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "d4f00e69d1a7436589af0eaf8bbc52cd"
format_stamp: "Formatted at 2026-08-09 02:18:51 on dist-test-slave-qg60"
server_key: "b9f4194ab568d115caa605f83cca8d50"
server_key_iv: "188267f1e4aaa555b7e9564528970e98"
server_key_version: "encryptionkey@0"
I20260809 02:18:51.969321 368 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:51.981077 368 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:51.983053 368 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:51.983210 368 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:51.987392 368 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40079
I20260809 02:18:51.987491 442 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40079 every 8 connection(s)
I20260809 02:18:51.987807 368 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:51.988777 443 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:51.991007 443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd: Bootstrap starting.
I20260809 02:18:51.991556 443 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:51.991797 443 log.cc:824] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:51.992443 443 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd: No bootstrap required, opened a new log
I20260809 02:18:51.993330 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 368
I20260809 02:18:51.993456 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:51.993687 9377 external_mini_cluster.cc:1428] Setting key 93de33609f42fb3fe08c2fd216e0a77a
I20260809 02:18:51.994689 443 raft_consensus.cc:348] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260809 02:18:51.994858 443 raft_consensus.cc:374] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:51.994899 443 raft_consensus.cc:729] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d4f00e69d1a7436589af0eaf8bbc52cd, State: Initialized, Role: FOLLOWER
I20260809 02:18:51.995095 443 consensus_queue.cc:260] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260809 02:18:51.995198 443 raft_consensus.cc:388] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:51.995244 443 raft_consensus.cc:482] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:51.995318 443 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:51.995862 443 raft_consensus.cc:504] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260809 02:18:51.995985 443 leader_election.cc:302] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: d4f00e69d1a7436589af0eaf8bbc52cd; no voters:
I20260809 02:18:51.996181 443 leader_election.cc:288] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:51.996205 448 raft_consensus.cc:2781] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:51.996321 448 raft_consensus.cc:686] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [term 1 LEADER]: Becoming Leader. State: Replica: d4f00e69d1a7436589af0eaf8bbc52cd, State: Running, Role: LEADER
I20260809 02:18:51.996459 443 sys_catalog.cc:564] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:51.996466 448 consensus_queue.cc:237] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } }
I20260809 02:18:51.996997 450 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [sys.catalog]: SysCatalogTable state changed. Reason: New leader d4f00e69d1a7436589af0eaf8bbc52cd. Latest consensus state: current_term: 1 leader_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } } }
I20260809 02:18:51.997092 450 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:51.997308 449 sys_catalog.cc:455] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "d4f00e69d1a7436589af0eaf8bbc52cd" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40079 } } }
I20260809 02:18:51.998045 449 sys_catalog.cc:458] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:51.998090 456 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:51.998567 456 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:52.000043 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:51.994225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:40647 (local address 127.0.0.1:40079)
0809 02:18:51.994413 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:51.994420 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:51.994445 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:18:51.994666 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:51.994672 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:51.994720 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:51.994828 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:51.995079 (+ 251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:51.996414 (+ 1335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:51.997559 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:51.998157 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:51.998262 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:51.999012 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:51.999016 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:51.999016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:51.999215 (+ 199us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:51.999261 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:51.999276 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:51.999741 (+ 465us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:51.999787 (+ 46us) server_negotiation.cc:299] Negotiation successful
0809 02:18:51.999815 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":70,"threads_started":1}
I20260809 02:18:52.000258 456 catalog_manager.cc:1347] Generated new cluster ID: dd63c350810747958a55bf86d72ffc5a
I20260809 02:18:52.000315 456 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:52.000974 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:52.007705 456 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:52.007931 447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:52.004831 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:48667 (local address 127.0.0.1:40079)
0809 02:18:52.004891 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:52.004893 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:52.004901 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:18:52.004922 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:52.004923 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:52.004934 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:52.005005 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:52.005155 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:52.005833 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:52.006574 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:52.006809 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:52.006882 (+ 73us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:52.007637 (+ 755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:52.007638 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:52.007639 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:52.007644 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:52.007660 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:52.007666 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:52.007722 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:52.007842 (+ 120us) server_negotiation.cc:299] Negotiation successful
0809 02:18:52.007856 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260809 02:18:52.008414 456 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:52.014324 456 catalog_manager.cc:5868] T 00000000000000000000000000000000 P d4f00e69d1a7436589af0eaf8bbc52cd: Generated new TSK 0
I20260809 02:18:52.014536 456 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:52.019519 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 368
W20260809 02:18:52.027534 353 connection.cc:535] client connection to 127.0.0.1:40079 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40079): Connection reset by peer (error 104)
2026-08-09T02:18:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1372 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:52 dist-test-slave-qg60 krb5kdc[476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:52 dist-test-slave-qg60 krb5kdc[476](info): set up 2 sockets
Aug 09 02:18:52 dist-test-slave-qg60 krb5kdc[476](info): commencing operation
krb5kdc: starting...
W20260809 02:18:53.130635 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.089s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:53 dist-test-slave-qg60 krb5kdc[476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260809 02:18:53.263471 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33091
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:53.349042 491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:53.349277 491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:53.349313 491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:53.350468 491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:53.350536 491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:53.350560 491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:53.350582 491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:53.350610 491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:53.352082 491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33091
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:53.352367 491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:53.352597 491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:53.357692 496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.357844 491 server_base.cc:1034] running on GCE node
W20260809 02:18:53.357887 497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:53.357702 501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:53.358202 491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:53.358498 491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:53.359684 491 hybrid_clock.cc:648] HybridClock initialized: now 1786241933359668 us; error 37 us; skew 500 ppm
Aug 09 02:18:53 dist-test-slave-qg60 krb5kdc[476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241933, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:53.362417 491 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260809 02:18:53.362900 491 webserver.cc:460] Webserver started at http://127.0.0.1:45429/ using document root <none> and password file <none>
I20260809 02:18:53.363138 491 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:53.363185 491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:53.363348 491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:53.364419 491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "0da6f351b47c49e2862956870c63f985"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-qg60"
server_key: "3808b763196f8a99167861a261999c1a"
server_key_iv: "cfb784a4139001c0976d81b6409412c8"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.364849 491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "0da6f351b47c49e2862956870c63f985"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-qg60"
server_key: "3808b763196f8a99167861a261999c1a"
server_key_iv: "cfb784a4139001c0976d81b6409412c8"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.366377 491 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:53.367480 508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:53.367666 491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:53.367748 491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "0da6f351b47c49e2862956870c63f985"
format_stamp: "Formatted at 2026-08-09 02:18:53 on dist-test-slave-qg60"
server_key: "3808b763196f8a99167861a261999c1a"
server_key_iv: "cfb784a4139001c0976d81b6409412c8"
server_key_version: "encryptionkey@0"
I20260809 02:18:53.367803 491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:53.371099 491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:53.373328 491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:53.373437 491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:53.377326 491 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39919
I20260809 02:18:53.377373 564 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39919 every 8 connection(s)
I20260809 02:18:53.377699 491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:53.378878 565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:53.379000 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 491
I20260809 02:18:53.379127 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:53.379377 9377 external_mini_cluster.cc:1428] Setting key 12229d493345a0b33c524b884bb3b630
I20260809 02:18:53.381970 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.379934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:55819 (local address 127.0.0.1:39919)
0809 02:18:53.380143 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.380151 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.380177 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:18:53.380651 (+ 474us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.380658 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.380682 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.380846 (+ 164us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.381525 (+ 679us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.381528 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.381528 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:53.381532 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.381569 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:53.381579 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.381640 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.381727 (+ 87us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.381761 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":77,"threads_started":1}
I20260809 02:18:53.383045 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:53.383065 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985: Bootstrap starting.
I20260809 02:18:53.383870 565 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:53.384253 565 log.cc:824] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:53.385099 565 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985: No bootstrap required, opened a new log
I20260809 02:18:53.386082 568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:53.384964 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.24:52199 (local address 127.0.0.1:39919)
0809 02:18:53.385008 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:53.385011 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:53.385148 (+ 137us) server_negotiation.cc:407] Connection header received
0809 02:18:53.385226 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:53.385227 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:53.385233 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:53.385301 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:53.385878 (+ 577us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:53.385879 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:53.385879 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:53.385884 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:53.385899 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:53.385903 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:53.385970 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:53.386011 (+ 41us) server_negotiation.cc:299] Negotiation successful
0809 02:18:53.386026 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:18:53.387533 565 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260809 02:18:53.387719 565 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:53.387770 565 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0da6f351b47c49e2862956870c63f985, State: Initialized, Role: FOLLOWER
I20260809 02:18:53.387895 565 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260809 02:18:53.387984 565 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:53.388031 565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:53.388078 565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:53.388696 565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260809 02:18:53.388844 565 leader_election.cc:302] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0da6f351b47c49e2862956870c63f985; no voters:
I20260809 02:18:53.389048 565 leader_election.cc:288] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:53.389164 577 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:53.389315 565 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:53.389348 577 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [term 1 LEADER]: Becoming Leader. State: Replica: 0da6f351b47c49e2862956870c63f985, State: Running, Role: LEADER
I20260809 02:18:53.389443 577 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } }
I20260809 02:18:53.389853 578 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0da6f351b47c49e2862956870c63f985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260809 02:18:53.389976 578 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:53.390165 579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0da6f351b47c49e2862956870c63f985. Latest consensus state: current_term: 1 leader_uuid: "0da6f351b47c49e2862956870c63f985" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0da6f351b47c49e2862956870c63f985" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39919 } } }
I20260809 02:18:53.390195 586 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:53.390228 579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:53.390745 586 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:53.392388 586 catalog_manager.cc:1347] Generated new cluster ID: 96321146fc7b4dbaade38430c84651f6
I20260809 02:18:53.392452 586 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:53.405402 586 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:53.405616 586 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:53.408295 586 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0da6f351b47c49e2862956870c63f985: Generated new TSK 0
I20260809 02:18:53.408492 586 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:53.410091 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 491
2026-08-09T02:18:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:53 dist-test-slave-qg60 krb5kdc[598](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:53 dist-test-slave-qg60 krb5kdc[598](info): set up 2 sockets
Aug 09 02:18:53 dist-test-slave-qg60 krb5kdc[598](info): commencing operation
krb5kdc: starting...
W20260809 02:18:54.505753 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.075s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:54.631902 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38597
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:37525
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38597
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:54.714960 613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:54.715215 613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:54.715253 613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:54.716390 613 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:54.716454 613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:54.716476 613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:54.716499 613 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:54.716527 613 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:54.718017 613 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:37525
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38597
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38597
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:54.718299 613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:54.718524 613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:54.723305 620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:54.723313 619 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:54.723313 622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:54.723660 613 server_base.cc:1034] running on GCE node
I20260809 02:18:54.723935 613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:54.724258 613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:54.725419 613 hybrid_clock.cc:648] HybridClock initialized: now 1786241934725401 us; error 34 us; skew 500 ppm
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[598](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241934, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:54.733587 613 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:54.734092 613 webserver.cc:460] Webserver started at http://127.9.40.126:33137/ using document root <none> and password file <none>
I20260809 02:18:54.734328 613 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:54.734398 613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:54.734551 613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:54.735716 613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "bea5248f83e64bc6a9d8d3949cbf8da9"
format_stamp: "Formatted at 2026-08-09 02:18:54 on dist-test-slave-qg60"
server_key: "9897baa8d83029356ddae29cbba0be8c"
server_key_iv: "044a27d894a26518e30339c97880f342"
server_key_version: "encryptionkey@0"
I20260809 02:18:54.736104 613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bea5248f83e64bc6a9d8d3949cbf8da9"
format_stamp: "Formatted at 2026-08-09 02:18:54 on dist-test-slave-qg60"
server_key: "9897baa8d83029356ddae29cbba0be8c"
server_key_iv: "044a27d894a26518e30339c97880f342"
server_key_version: "encryptionkey@0"
I20260809 02:18:54.737715 613 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:18:54.738590 629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:54.738834 613 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:54.738945 613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "bea5248f83e64bc6a9d8d3949cbf8da9"
format_stamp: "Formatted at 2026-08-09 02:18:54 on dist-test-slave-qg60"
server_key: "9897baa8d83029356ddae29cbba0be8c"
server_key_iv: "044a27d894a26518e30339c97880f342"
server_key_version: "encryptionkey@0"
I20260809 02:18:54.739045 613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:54.755694 613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:54.757918 613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:54.758080 613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:54.762188 613 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38597
I20260809 02:18:54.762250 687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38597 every 8 connection(s)
I20260809 02:18:54.762601 613 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:54.763535 688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:54.765882 688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: Bootstrap starting.
I20260809 02:18:54.766436 688 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:54.766690 688 log.cc:824] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:54.767373 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 613
I20260809 02:18:54.767386 688 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: No bootstrap required, opened a new log
I20260809 02:18:54.767501 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:54.767690 9377 external_mini_cluster.cc:1428] Setting key b2bd9082f21a031f47f0c8b6918a94a6
I20260809 02:18:54.769788 688 raft_consensus.cc:348] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } }
I20260809 02:18:54.769969 688 raft_consensus.cc:374] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:54.770007 688 raft_consensus.cc:729] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bea5248f83e64bc6a9d8d3949cbf8da9, State: Initialized, Role: FOLLOWER
I20260809 02:18:54.770180 688 consensus_queue.cc:260] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } }
I20260809 02:18:54.770247 688 raft_consensus.cc:388] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:54.770269 688 raft_consensus.cc:482] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:54.770336 688 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:54.771452 688 raft_consensus.cc:504] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } }
I20260809 02:18:54.771565 688 leader_election.cc:302] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bea5248f83e64bc6a9d8d3949cbf8da9; no voters:
I20260809 02:18:54.771732 688 leader_election.cc:288] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:54.771780 693 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:54.771924 693 raft_consensus.cc:686] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [term 1 LEADER]: Becoming Leader. State: Replica: bea5248f83e64bc6a9d8d3949cbf8da9, State: Running, Role: LEADER
I20260809 02:18:54.771975 688 sys_catalog.cc:564] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:54.772060 693 consensus_queue.cc:237] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } }
I20260809 02:18:54.772512 696 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bea5248f83e64bc6a9d8d3949cbf8da9. Latest consensus state: current_term: 1 leader_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } } }
I20260809 02:18:54.772688 696 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:54.773434 697 sys_catalog.cc:455] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bea5248f83e64bc6a9d8d3949cbf8da9" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38597 } } }
I20260809 02:18:54.773586 697 sys_catalog.cc:458] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9 [sys.catalog]: This master's current role is: LEADER
W20260809 02:18:54.774008 713 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:54.774071 713 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:54.774160 706 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:54.774329 706 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:54.776502 706 catalog_manager.cc:1347] Generated new cluster ID: b447d7e702fd4db283db78bf97929cd5
I20260809 02:18:54.776561 706 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[598](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:54.781243 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:54.768235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60157 (local address 127.9.40.126:38597)
0809 02:18:54.768457 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:54.768466 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:54.768503 (+ 37us) server_negotiation.cc:407] Connection header received
0809 02:18:54.769743 (+ 1240us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:54.769752 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:54.769826 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:54.769976 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:54.770775 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:54.772274 (+ 1499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:54.773471 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:54.774122 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:54.774227 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:54.777277 (+ 3050us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:54.777281 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:54.777281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:54.777300 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:54.779371 (+ 2071us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:54.779826 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:54.779830 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:54.779830 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:54.779971 (+ 141us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:54.780210 (+ 239us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:54.780211 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:54.780211 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:54.780543 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:54.780778 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:54.780884 (+ 106us) server_negotiation.cc:299] Negotiation successful
0809 02:18:54.781014 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":168,"thread_start_us":117,"threads_started":1}
I20260809 02:18:54.782313 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:54.791467 692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:54.783901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51743 (local address 127.9.40.126:38597)
0809 02:18:54.783935 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:54.783937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:54.784059 (+ 122us) server_negotiation.cc:407] Connection header received
0809 02:18:54.784122 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:54.784122 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:54.784131 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:54.784180 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:54.784834 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:54.785579 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:54.786616 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:54.786939 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:54.787002 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:54.788390 (+ 1388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:54.788391 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:54.788391 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:54.788398 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:54.790038 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:54.790455 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:54.790456 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:54.790457 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:54.790513 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:54.790750 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:54.790751 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:54.790751 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:54.790866 (+ 115us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:54.790967 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:54.791277 (+ 310us) server_negotiation.cc:299] Negotiation successful
0809 02:18:54.791360 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260809 02:18:54.803747 706 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:54.804308 706 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:54.817123 706 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bea5248f83e64bc6a9d8d3949cbf8da9: Generated new TSK 0
I20260809 02:18:54.817342 706 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:54.823158 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 613
W20260809 02:18:54.831094 596 connection.cc:535] client connection to 127.9.40.126:38597 recv error: Network error: failed to read from TLS socket (remote: 127.9.40.126:38597): Connection reset by peer (error 104)
2026-08-09T02:18:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1414 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[724](info): set up 2 sockets
Aug 09 02:18:54 dist-test-slave-qg60 krb5kdc[724](info): commencing operation
krb5kdc: starting...
W20260809 02:18:55.967377 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.122s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:56 dist-test-slave-qg60 krb5kdc[724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:56.096392 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36913
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38679
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:36913
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:56.201393 738 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:56.201648 738 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:56.201689 738 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:56.202904 738 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:56.202971 738 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:56.202998 738 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:56.203028 738 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:56.203058 738 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:56.204568 738 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38679
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:36913
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:36913
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.738
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:56.204877 738 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:56.205108 738 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260809 02:18:56.210035 738 server_base.cc:1034] running on GCE node
W20260809 02:18:56.210017 745 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:56.210005 743 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:56.210011 747 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:56.210508 738 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:56.210808 738 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:56.211947 738 hybrid_clock.cc:648] HybridClock initialized: now 1786241936211940 us; error 33 us; skew 500 ppm
Aug 09 02:18:56 dist-test-slave-qg60 krb5kdc[724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241936, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:56.214500 738 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:56.214967 738 webserver.cc:460] Webserver started at http://127.9.40.126:46843/ using document root <none> and password file <none>
I20260809 02:18:56.215201 738 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:56.215269 738 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:56.215415 738 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:56.216516 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "49d03c197ec74fae9cf2b96628940fdc"
format_stamp: "Formatted at 2026-08-09 02:18:56 on dist-test-slave-qg60"
server_key: "02e720ab4f7916e4cadd113283d5b3c1"
server_key_iv: "b6ba9abc03af7a07221cc9b6e0d118b0"
server_key_version: "encryptionkey@0"
I20260809 02:18:56.216923 738 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "49d03c197ec74fae9cf2b96628940fdc"
format_stamp: "Formatted at 2026-08-09 02:18:56 on dist-test-slave-qg60"
server_key: "02e720ab4f7916e4cadd113283d5b3c1"
server_key_iv: "b6ba9abc03af7a07221cc9b6e0d118b0"
server_key_version: "encryptionkey@0"
I20260809 02:18:56.218290 738 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:18:56.219049 754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:56.219211 738 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:18:56.219312 738 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "49d03c197ec74fae9cf2b96628940fdc"
format_stamp: "Formatted at 2026-08-09 02:18:56 on dist-test-slave-qg60"
server_key: "02e720ab4f7916e4cadd113283d5b3c1"
server_key_iv: "b6ba9abc03af7a07221cc9b6e0d118b0"
server_key_version: "encryptionkey@0"
I20260809 02:18:56.219393 738 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:56.269532 738 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:56.271930 738 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:56.272317 738 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:56.277132 738 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:36913
I20260809 02:18:56.277169 811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:36913 every 8 connection(s)
I20260809 02:18:56.277645 738 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:56.278761 812 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:56.281404 812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc: Bootstrap starting.
I20260809 02:18:56.282065 812 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:56.282387 812 log.cc:824] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:56.283181 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 738
I20260809 02:18:56.283303 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:56.283327 812 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc: No bootstrap required, opened a new log
I20260809 02:18:56.283509 9377 external_mini_cluster.cc:1428] Setting key 28cd0a8165533ccee0f73b18a9ff99eb
I20260809 02:18:56.286610 812 raft_consensus.cc:348] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } }
I20260809 02:18:56.286803 812 raft_consensus.cc:374] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:56.286842 812 raft_consensus.cc:729] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 49d03c197ec74fae9cf2b96628940fdc, State: Initialized, Role: FOLLOWER
I20260809 02:18:56.286975 812 consensus_queue.cc:260] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } }
I20260809 02:18:56.287055 812 raft_consensus.cc:388] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:56.287084 812 raft_consensus.cc:482] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:56.287132 812 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:56.287851 812 raft_consensus.cc:504] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } }
I20260809 02:18:56.287950 812 leader_election.cc:302] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 49d03c197ec74fae9cf2b96628940fdc; no voters:
I20260809 02:18:56.288189 812 leader_election.cc:288] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:56.288216 818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:56.288339 818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [term 1 LEADER]: Becoming Leader. State: Replica: 49d03c197ec74fae9cf2b96628940fdc, State: Running, Role: LEADER
I20260809 02:18:56.288443 818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } }
I20260809 02:18:56.288488 812 sys_catalog.cc:564] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:56.288823 818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "49d03c197ec74fae9cf2b96628940fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } } }
I20260809 02:18:56.288899 818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:56.289125 818 sys_catalog.cc:455] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 49d03c197ec74fae9cf2b96628940fdc. Latest consensus state: current_term: 1 leader_uuid: "49d03c197ec74fae9cf2b96628940fdc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "49d03c197ec74fae9cf2b96628940fdc" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 36913 } } }
I20260809 02:18:56.289206 818 sys_catalog.cc:458] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:56.289776 824 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:56.290051 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:56.284145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57329 (local address 127.9.40.126:36913)
0809 02:18:56.284347 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:56.284356 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:56.284397 (+ 41us) server_negotiation.cc:407] Connection header received
0809 02:18:56.285025 (+ 628us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:56.285031 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:56.285087 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:56.285200 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:56.285457 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:56.286309 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:56.287554 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:56.288000 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:56.288184 (+ 184us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:56.289063 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:56.289067 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:56.289068 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:56.289371 (+ 303us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:56.289411 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:56.289427 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:56.289756 (+ 329us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:56.289803 (+ 47us) server_negotiation.cc:299] Negotiation successful
0809 02:18:56.289832 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":118,"threads_started":1}
I20260809 02:18:56.290619 824 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:56.291180 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:56.292309 824 catalog_manager.cc:1347] Generated new cluster ID: 5eb79fb7d075459f9cc78fb90c6578f9
I20260809 02:18:56.292367 824 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:56.297505 817 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:56.293177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41309 (local address 127.9.40.126:36913)
0809 02:18:56.293226 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:56.293228 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:56.293346 (+ 118us) server_negotiation.cc:407] Connection header received
0809 02:18:56.293433 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:56.293434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:56.293442 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:56.293469 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:56.293822 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:56.294646 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:56.295789 (+ 1143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:56.296117 (+ 328us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:56.296160 (+ 43us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:56.297251 (+ 1091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:56.297252 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:56.297253 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:56.297257 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:56.297272 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:56.297276 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:56.297305 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:56.297416 (+ 111us) server_negotiation.cc:299] Negotiation successful
0809 02:18:56.297428 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260809 02:18:56.306398 824 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:56.307372 824 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:56.311828 824 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 49d03c197ec74fae9cf2b96628940fdc: Generated new TSK 0
I20260809 02:18:56.312039 824 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:56.318872 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 738
W20260809 02:18:56.325824 722 connection.cc:535] client connection to 127.9.40.126:36913 recv error: Network error: failed to read from TLS socket (remote: 127.9.40.126:36913): Connection reset by peer (error 104)
2026-08-09T02:18:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1494 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:56 dist-test-slave-qg60 krb5kdc[849](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:56 dist-test-slave-qg60 krb5kdc[849](info): set up 2 sockets
Aug 09 02:18:56 dist-test-slave-qg60 krb5kdc[849](info): commencing operation
krb5kdc: starting...
W20260809 02:18:57.434583 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.095s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:57 dist-test-slave-qg60 krb5kdc[849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241937, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:57.567680 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35693
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:45937
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35693
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:57.653000 863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:57.653285 863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:57.653324 863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:57.654536 863 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:57.654618 863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:57.654644 863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:57.654667 863 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:57.654695 863 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:57.656307 863 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:45937
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35693
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35693
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:57.656623 863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:57.656885 863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:57.662101 871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:57.662266 863 server_base.cc:1034] running on GCE node
W20260809 02:18:57.662173 869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:57.662130 868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:57.662655 863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:57.662995 863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:57.664153 863 hybrid_clock.cc:648] HybridClock initialized: now 1786241937664128 us; error 38 us; skew 500 ppm
Aug 09 02:18:57 dist-test-slave-qg60 krb5kdc[849](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241937, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:57.666918 863 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:57.667426 863 webserver.cc:460] Webserver started at http://127.9.40.126:42531/ using document root <none> and password file <none>
I20260809 02:18:57.667667 863 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:57.667716 863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:57.667889 863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:57.669095 863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8"
format_stamp: "Formatted at 2026-08-09 02:18:57 on dist-test-slave-qg60"
server_key: "2d6e4d8cb568674efdc14a5b011b6921"
server_key_iv: "7bdc3758084cd3c67a84153cd06c0f76"
server_key_version: "encryptionkey@0"
I20260809 02:18:57.669461 863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8"
format_stamp: "Formatted at 2026-08-09 02:18:57 on dist-test-slave-qg60"
server_key: "2d6e4d8cb568674efdc14a5b011b6921"
server_key_iv: "7bdc3758084cd3c67a84153cd06c0f76"
server_key_version: "encryptionkey@0"
I20260809 02:18:57.670876 863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:18:57.671669 879 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:57.671875 863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:57.671952 863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8"
format_stamp: "Formatted at 2026-08-09 02:18:57 on dist-test-slave-qg60"
server_key: "2d6e4d8cb568674efdc14a5b011b6921"
server_key_iv: "7bdc3758084cd3c67a84153cd06c0f76"
server_key_version: "encryptionkey@0"
I20260809 02:18:57.672055 863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:57.675619 863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:57.678339 863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:57.678488 863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:57.682786 863 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35693
I20260809 02:18:57.682859 936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35693 every 8 connection(s)
I20260809 02:18:57.683195 863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:57.683576 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 863
I20260809 02:18:57.683727 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:57.683949 9377 external_mini_cluster.cc:1428] Setting key 074467a69f424d64d7eb60712b31430b
I20260809 02:18:57.684391 937 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:57.687014 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:57.684616 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35039 (local address 127.9.40.126:35693)
0809 02:18:57.684849 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:57.684857 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:57.684882 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:18:57.685571 (+ 689us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:57.685579 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:57.685611 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:57.685795 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:57.686545 (+ 750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:57.686549 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:57.686549 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:57.686553 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:57.686600 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:57.686611 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:57.686716 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:57.686811 (+ 95us) server_negotiation.cc:299] Negotiation successful
0809 02:18:57.686854 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":173,"thread_start_us":125,"threads_started":1}
I20260809 02:18:57.688077 937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8: Bootstrap starting.
I20260809 02:18:57.688094 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:57.688791 937 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:57.689085 937 log.cc:824] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:57.689941 937 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8: No bootstrap required, opened a new log
I20260809 02:18:57.690917 940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:57.689682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35291 (local address 127.9.40.126:35693)
0809 02:18:57.689718 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:57.689724 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:57.689871 (+ 147us) server_negotiation.cc:407] Connection header received
0809 02:18:57.689946 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:57.689947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:57.689952 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:57.690023 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:57.690699 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:57.690700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:57.690700 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:18:57.690705 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:57.690719 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:18:57.690722 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:57.690780 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:57.690837 (+ 57us) server_negotiation.cc:299] Negotiation successful
0809 02:18:57.690851 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260809 02:18:57.692325 937 raft_consensus.cc:348] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } }
I20260809 02:18:57.692471 937 raft_consensus.cc:374] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:57.692494 937 raft_consensus.cc:729] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb6f4fccbc34e56be2e1a5c6c90d5e8, State: Initialized, Role: FOLLOWER
I20260809 02:18:57.692626 937 consensus_queue.cc:260] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } }
I20260809 02:18:57.692713 937 raft_consensus.cc:388] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:57.692799 937 raft_consensus.cc:482] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:57.692854 937 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:57.693418 937 raft_consensus.cc:504] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } }
I20260809 02:18:57.693519 937 leader_election.cc:302] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afb6f4fccbc34e56be2e1a5c6c90d5e8; no voters:
I20260809 02:18:57.693663 937 leader_election.cc:288] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:57.693748 947 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:57.693908 937 sys_catalog.cc:564] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:57.693943 947 raft_consensus.cc:686] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [term 1 LEADER]: Becoming Leader. State: Replica: afb6f4fccbc34e56be2e1a5c6c90d5e8, State: Running, Role: LEADER
I20260809 02:18:57.694025 947 consensus_queue.cc:237] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } }
I20260809 02:18:57.694442 949 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afb6f4fccbc34e56be2e1a5c6c90d5e8. Latest consensus state: current_term: 1 leader_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } } }
I20260809 02:18:57.694566 949 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:57.694427 948 sys_catalog.cc:455] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afb6f4fccbc34e56be2e1a5c6c90d5e8" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35693 } } }
I20260809 02:18:57.694682 948 sys_catalog.cc:458] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:57.694809 956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:57.695363 956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:57.697163 956 catalog_manager.cc:1347] Generated new cluster ID: a756d52781614a0fb92c525f01a611e8
I20260809 02:18:57.697239 956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:18:57.709496 956 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:57.709748 956 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:57.714099 956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afb6f4fccbc34e56be2e1a5c6c90d5e8: Generated new TSK 0
I20260809 02:18:57.714330 956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:57.721073 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 863
2026-08-09T02:18:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:57 dist-test-slave-qg60 krb5kdc[969](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:57 dist-test-slave-qg60 krb5kdc[969](info): set up 2 sockets
Aug 09 02:18:57 dist-test-slave-qg60 krb5kdc[969](info): commencing operation
krb5kdc: starting...
W20260809 02:18:58.834131 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.091s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:18:58 dist-test-slave-qg60 krb5kdc[969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:18:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:18:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:18:58.968713 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41845
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38529
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:41845
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:18:59.052186 982 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:18:59.052441 982 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:18:59.052480 982 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:18:59.053644 982 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:18:59.053718 982 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:18:59.053742 982 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:18:59.053774 982 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:18:59.053805 982 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:18:59.055291 982 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38529
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:41845
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:41845
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.982
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:18:59.055572 982 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:18:59.055809 982 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:18:59.060859 992 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:18:59.060842 987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:59.060930 982 server_base.cc:1034] running on GCE node
W20260809 02:18:59.060842 988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:18:59.061357 982 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:18:59.061657 982 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:18:59.062826 982 hybrid_clock.cc:648] HybridClock initialized: now 1786241939062811 us; error 26 us; skew 500 ppm
Aug 09 02:18:59 dist-test-slave-qg60 krb5kdc[969](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241939, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:18:59.065502 982 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:18:59.065984 982 webserver.cc:460] Webserver started at http://127.9.40.126:43935/ using document root <none> and password file <none>
I20260809 02:18:59.066171 982 fs_manager.cc:359] Metadata directory not provided
I20260809 02:18:59.066211 982 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:18:59.066304 982 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:18:59.067343 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "5953784560834a9bad0ecc41c91252c7"
format_stamp: "Formatted at 2026-08-09 02:18:59 on dist-test-slave-qg60"
server_key: "07e180423d79c26a42e8a0abfb5dbeb9"
server_key_iv: "baaa2094f2dfe342578f14424a460fc4"
server_key_version: "encryptionkey@0"
I20260809 02:18:59.067678 982 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "5953784560834a9bad0ecc41c91252c7"
format_stamp: "Formatted at 2026-08-09 02:18:59 on dist-test-slave-qg60"
server_key: "07e180423d79c26a42e8a0abfb5dbeb9"
server_key_iv: "baaa2094f2dfe342578f14424a460fc4"
server_key_version: "encryptionkey@0"
I20260809 02:18:59.069090 982 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:18:59.069834 999 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:18:59.070093 982 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:18:59.070181 982 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "5953784560834a9bad0ecc41c91252c7"
format_stamp: "Formatted at 2026-08-09 02:18:59 on dist-test-slave-qg60"
server_key: "07e180423d79c26a42e8a0abfb5dbeb9"
server_key_iv: "baaa2094f2dfe342578f14424a460fc4"
server_key_version: "encryptionkey@0"
I20260809 02:18:59.070250 982 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:18:59.080618 982 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:18:59.083082 982 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:18:59.083201 982 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:18:59.087255 982 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:41845
I20260809 02:18:59.087309 1060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:41845 every 8 connection(s)
I20260809 02:18:59.087659 982 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:18:59.088575 1061 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:18:59.090958 1061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: Bootstrap starting.
I20260809 02:18:59.091590 1061 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: Neither blocks nor log segments found. Creating new log.
I20260809 02:18:59.091910 1061 log.cc:824] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: Log is configured to *not* fsync() on all Append() calls
I20260809 02:18:59.092623 1061 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: No bootstrap required, opened a new log
I20260809 02:18:59.094311 1061 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } }
I20260809 02:18:59.094483 1061 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:18:59.094478 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 982
I20260809 02:18:59.094532 1061 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5953784560834a9bad0ecc41c91252c7, State: Initialized, Role: FOLLOWER
I20260809 02:18:59.094633 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:18:59.094722 1061 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } }
I20260809 02:18:59.094833 1061 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:18:59.094880 1061 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:18:59.094892 9377 external_mini_cluster.cc:1428] Setting key 2dcbaa681753e84068c28a81d1779493
I20260809 02:18:59.094935 1061 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:18:59.096208 1061 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } }
I20260809 02:18:59.096321 1061 leader_election.cc:302] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5953784560834a9bad0ecc41c91252c7; no voters:
I20260809 02:18:59.096494 1061 leader_election.cc:288] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:18:59.096518 1066 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:18:59.096642 1066 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [term 1 LEADER]: Becoming Leader. State: Replica: 5953784560834a9bad0ecc41c91252c7, State: Running, Role: LEADER
I20260809 02:18:59.096748 1066 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } }
I20260809 02:18:59.096787 1061 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:18:59.097113 1067 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5953784560834a9bad0ecc41c91252c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } } }
I20260809 02:18:59.097292 1067 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [sys.catalog]: This master's current role is: LEADER
I20260809 02:18:59.097126 1066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5953784560834a9bad0ecc41c91252c7. Latest consensus state: current_term: 1 leader_uuid: "5953784560834a9bad0ecc41c91252c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5953784560834a9bad0ecc41c91252c7" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 41845 } } }
I20260809 02:18:59.098407 1066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7 [sys.catalog]: This master's current role is: LEADER
W20260809 02:18:59.098734 1082 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:18:59.098809 1082 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:18:59.098865 1074 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:18:59.099097 1074 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:18:59.101311 1074 catalog_manager.cc:1347] Generated new cluster ID: 9da7ce8c296c4a629723ca60b883baf3
I20260809 02:18:59.101378 1074 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Aug 09 02:18:59 dist-test-slave-qg60 krb5kdc[969](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241938, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:18:59.107692 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:59.095570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54975 (local address 127.9.40.126:41845)
0809 02:18:59.095812 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:59.095839 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:59.095890 (+ 51us) server_negotiation.cc:407] Connection header received
0809 02:18:59.096025 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:59.096033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:59.096092 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:59.096249 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:59.097187 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:59.098777 (+ 1590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:59.099879 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:59.100453 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:59.100561 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:59.103530 (+ 2969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:59.103535 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:59.103535 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:59.103548 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:59.105801 (+ 2253us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:59.106210 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:59.106213 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:59.106214 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:59.106345 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:59.106562 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:59.106563 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:59.106563 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:59.106896 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:59.107066 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:59.107176 (+ 110us) server_negotiation.cc:299] Negotiation successful
0809 02:18:59.107456 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":109,"threads_started":1}
I20260809 02:18:59.108057 1074 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:18:59.108806 1074 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:18:59.109045 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:18:59.113546 1074 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5953784560834a9bad0ecc41c91252c7: Generated new TSK 0
I20260809 02:18:59.113714 1074 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:18:59.117916 1065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:18:59.110804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58097 (local address 127.9.40.126:41845)
0809 02:18:59.110843 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:18:59.110845 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:18:59.110988 (+ 143us) server_negotiation.cc:407] Connection header received
0809 02:18:59.111031 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:18:59.111032 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:18:59.111058 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:18:59.111107 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:18:59.111756 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:59.112527 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:59.113628 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:18:59.113921 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:18:59.113979 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:18:59.115359 (+ 1380us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:18:59.115360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:18:59.115361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:18:59.115367 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:18:59.116877 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:59.117243 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:59.117244 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:59.117244 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:59.117293 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:18:59.117481 (+ 188us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:18:59.117481 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:18:59.117482 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:18:59.117571 (+ 89us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:18:59.117661 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0809 02:18:59.117798 (+ 137us) server_negotiation.cc:299] Negotiation successful
0809 02:18:59.117859 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:18:59.119947 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 982
2026-08-09T02:18:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1400 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:18:59 dist-test-slave-qg60 krb5kdc[1094](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:18:59 dist-test-slave-qg60 krb5kdc[1094](info): set up 2 sockets
Aug 09 02:18:59 dist-test-slave-qg60 krb5kdc[1094](info): commencing operation
krb5kdc: starting...
W20260809 02:19:00.213410 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:19:00 dist-test-slave-qg60 krb5kdc[1094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:19:00.340289 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38465
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:38221
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:00.424463 1110 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:00.424698 1110 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:00.424746 1110 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:00.425887 1110 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:00.425930 1110 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:00.425943 1110 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:00.425956 1110 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:00.425969 1110 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:00.427443 1110 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:38221
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1110
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:00.427667 1110 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:00.427860 1110 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:00.433044 1117 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:00.433037 1119 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:00.433144 1116 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:00.433071 1110 server_base.cc:1034] running on GCE node
I20260809 02:19:00.433547 1110 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:00.433866 1110 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:00.435074 1110 hybrid_clock.cc:648] HybridClock initialized: now 1786241940435061 us; error 28 us; skew 500 ppm
Aug 09 02:19:00 dist-test-slave-qg60 krb5kdc[1094](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241940, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:00.445433 1110 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:19:00.445981 1110 webserver.cc:460] Webserver started at http://127.9.40.126:33009/ using document root <none> and password file <none>
I20260809 02:19:00.446224 1110 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:00.446293 1110 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:00.446451 1110 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:00.447602 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "01987716f0214dc8ace098b7998cc18a"
format_stamp: "Formatted at 2026-08-09 02:19:00 on dist-test-slave-qg60"
server_key: "3240d0783f14bc1180642ffc10db1590"
server_key_iv: "7a9b141b3b69c93a3ea05b1d6ea9eb06"
server_key_version: "encryptionkey@0"
I20260809 02:19:00.447995 1110 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "01987716f0214dc8ace098b7998cc18a"
format_stamp: "Formatted at 2026-08-09 02:19:00 on dist-test-slave-qg60"
server_key: "3240d0783f14bc1180642ffc10db1590"
server_key_iv: "7a9b141b3b69c93a3ea05b1d6ea9eb06"
server_key_version: "encryptionkey@0"
I20260809 02:19:00.449509 1110 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:19:00.450311 1126 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:00.450541 1110 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:19:00.450654 1110 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "01987716f0214dc8ace098b7998cc18a"
format_stamp: "Formatted at 2026-08-09 02:19:00 on dist-test-slave-qg60"
server_key: "3240d0783f14bc1180642ffc10db1590"
server_key_iv: "7a9b141b3b69c93a3ea05b1d6ea9eb06"
server_key_version: "encryptionkey@0"
I20260809 02:19:00.450731 1110 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:00.463378 1110 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:00.465575 1110 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:00.465703 1110 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:00.470252 1110 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38465
I20260809 02:19:00.470449 1186 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38465 every 8 connection(s)
I20260809 02:19:00.470666 1110 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:00.471573 1187 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:00.473923 1187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a: Bootstrap starting.
I20260809 02:19:00.474460 1187 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:00.474717 1187 log.cc:824] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:00.475379 1187 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a: No bootstrap required, opened a new log
I20260809 02:19:00.476298 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1110
I20260809 02:19:00.476426 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:19:00.476653 9377 external_mini_cluster.cc:1428] Setting key 186afa52153e963baa4e05d63af13fba
I20260809 02:19:00.477654 1187 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } }
I20260809 02:19:00.477833 1187 raft_consensus.cc:374] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:00.477875 1187 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01987716f0214dc8ace098b7998cc18a, State: Initialized, Role: FOLLOWER
I20260809 02:19:00.478012 1187 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } }
I20260809 02:19:00.478111 1187 raft_consensus.cc:388] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:00.478135 1187 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:00.478202 1187 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:00.478761 1187 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } }
I20260809 02:19:00.478847 1187 leader_election.cc:302] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 01987716f0214dc8ace098b7998cc18a; no voters:
I20260809 02:19:00.479019 1187 leader_election.cc:288] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:00.479071 1194 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:00.479233 1194 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [term 1 LEADER]: Becoming Leader. State: Replica: 01987716f0214dc8ace098b7998cc18a, State: Running, Role: LEADER
I20260809 02:19:00.479283 1187 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:00.479368 1194 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } }
I20260809 02:19:00.479700 1194 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01987716f0214dc8ace098b7998cc18a. Latest consensus state: current_term: 1 leader_uuid: "01987716f0214dc8ace098b7998cc18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } } }
I20260809 02:19:00.479846 1194 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:00.479713 1195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01987716f0214dc8ace098b7998cc18a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "01987716f0214dc8ace098b7998cc18a" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38465 } } }
I20260809 02:19:00.480146 1195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:00.480479 1203 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:00.481053 1203 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:00.482667 1203 catalog_manager.cc:1347] Generated new cluster ID: 23c16986a628413fb879dfaedc7490ca
I20260809 02:19:00.482728 1203 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:00.483307 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:00.477268 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59385 (local address 127.9.40.126:38465)
0809 02:19:00.477454 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:00.477462 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:00.477487 (+ 25us) server_negotiation.cc:407] Connection header received
0809 02:19:00.477629 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:00.477635 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:00.477679 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:00.477788 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:00.478040 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:00.479597 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:00.480890 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:00.481523 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:00.481639 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:00.482413 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:00.482416 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:00.482416 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:00.482636 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:00.482676 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:00.482692 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:00.482762 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:00.483035 (+ 273us) server_negotiation.cc:299] Negotiation successful
0809 02:19:00.483098 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":98,"threads_started":1}
I20260809 02:19:00.484333 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:19:00.489729 1192 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:00.485958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49829 (local address 127.9.40.126:38465)
0809 02:19:00.486001 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:00.486003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:00.486108 (+ 105us) server_negotiation.cc:407] Connection header received
0809 02:19:00.486168 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:00.486168 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:00.486178 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:00.486216 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:00.486362 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:00.487065 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:00.488111 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:00.488368 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:00.488435 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:00.489229 (+ 794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:00.489231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:00.489231 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:00.489235 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:00.489251 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:00.489255 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:00.489316 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:00.489598 (+ 282us) server_negotiation.cc:299] Negotiation successful
0809 02:19:00.489651 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260809 02:19:00.489801 1203 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:00.490325 1203 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:00.498346 1203 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01987716f0214dc8ace098b7998cc18a: Generated new TSK 0
I20260809 02:19:00.498556 1203 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:00.501113 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1110
2026-08-09T02:19:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1379 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:00 dist-test-slave-qg60 krb5kdc[1223](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:00 dist-test-slave-qg60 krb5kdc[1223](info): set up 2 sockets
Aug 09 02:19:00 dist-test-slave-qg60 krb5kdc[1223](info): commencing operation
krb5kdc: starting...
W20260809 02:19:01.652980 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:19:01 dist-test-slave-qg60 krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241941, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:19:01.783830 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42807
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:44083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:42807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:01.869225 1237 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:01.869568 1237 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:01.869606 1237 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:01.870930 1237 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:01.870978 1237 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:01.870993 1237 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:01.871006 1237 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:01.871018 1237 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:01.872474 1237 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:44083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:42807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:42807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1237
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:01.872684 1237 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:01.872946 1237 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:01.878104 1244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:01.878223 1237 server_base.cc:1034] running on GCE node
W20260809 02:19:01.878074 1243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:01.878098 1246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:01.878572 1237 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:01.878880 1237 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:01.880086 1237 hybrid_clock.cc:648] HybridClock initialized: now 1786241941880073 us; error 44 us; skew 500 ppm
Aug 09 02:19:01 dist-test-slave-qg60 krb5kdc[1223](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241941, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:01.882710 1237 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:19:01.883206 1237 webserver.cc:460] Webserver started at http://127.9.40.126:40767/ using document root <none> and password file <none>
I20260809 02:19:01.883442 1237 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:01.883512 1237 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:01.883674 1237 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:01.884835 1237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "7d088ff0b8f740878af02f3bedac77b4"
format_stamp: "Formatted at 2026-08-09 02:19:01 on dist-test-slave-qg60"
server_key: "169f44bd8577332ef0debb5fdf8902fb"
server_key_iv: "871ebe5e41052e0b2d7b4f07af8c5b2e"
server_key_version: "encryptionkey@0"
I20260809 02:19:01.885229 1237 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "7d088ff0b8f740878af02f3bedac77b4"
format_stamp: "Formatted at 2026-08-09 02:19:01 on dist-test-slave-qg60"
server_key: "169f44bd8577332ef0debb5fdf8902fb"
server_key_iv: "871ebe5e41052e0b2d7b4f07af8c5b2e"
server_key_version: "encryptionkey@0"
I20260809 02:19:01.886638 1237 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:19:01.887483 1255 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:01.887698 1237 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260809 02:19:01.887809 1237 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "7d088ff0b8f740878af02f3bedac77b4"
format_stamp: "Formatted at 2026-08-09 02:19:01 on dist-test-slave-qg60"
server_key: "169f44bd8577332ef0debb5fdf8902fb"
server_key_iv: "871ebe5e41052e0b2d7b4f07af8c5b2e"
server_key_version: "encryptionkey@0"
I20260809 02:19:01.887888 1237 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:01.891497 1237 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:01.893818 1237 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:01.893985 1237 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:01.898218 1237 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:42807
I20260809 02:19:01.898257 1310 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:42807 every 8 connection(s)
I20260809 02:19:01.898623 1237 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:01.899473 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1237
I20260809 02:19:01.899641 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:19:01.899771 1311 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:01.899951 9377 external_mini_cluster.cc:1428] Setting key 3cb56e97af5d1904daf49175f5a328d1
I20260809 02:19:01.903823 1311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4: Bootstrap starting.
I20260809 02:19:01.904281 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:01.900517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50599 (local address 127.9.40.126:42807)
0809 02:19:01.902071 (+ 1554us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:01.902081 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:01.902119 (+ 38us) server_negotiation.cc:407] Connection header received
0809 02:19:01.902805 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:01.902814 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:01.902853 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:01.903018 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:01.903710 (+ 692us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:01.903714 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:01.903714 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:01.903719 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:01.903768 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:01.903783 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:01.903871 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:01.903907 (+ 36us) server_negotiation.cc:299] Negotiation successful
0809 02:19:01.904061 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1469,"thread_start_us":111,"threads_started":1}
I20260809 02:19:01.904667 1311 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:01.905085 1311 log.cc:824] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:01.905377 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:19:01.905929 1311 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4: No bootstrap required, opened a new log
I20260809 02:19:01.908071 1314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:01.906889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44293 (local address 127.9.40.126:42807)
0809 02:19:01.906946 (+ 57us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:01.906948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:01.907065 (+ 117us) server_negotiation.cc:407] Connection header received
0809 02:19:01.907170 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:01.907171 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:01.907175 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:01.907228 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:01.907863 (+ 635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:01.907864 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:01.907864 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0809 02:19:01.907868 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:01.907883 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0809 02:19:01.907887 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:01.907934 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:01.907988 (+ 54us) server_negotiation.cc:299] Negotiation successful
0809 02:19:01.908003 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:01.908648 1311 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } }
I20260809 02:19:01.908872 1311 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260809 02:19:01.908962 1311 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d088ff0b8f740878af02f3bedac77b4, State: Initialized, Role: FOLLOWER
I20260809 02:19:01.909144 1311 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } }
I20260809 02:19:01.909250 1311 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260809 02:19:01.909319 1311 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260809 02:19:01.909379 1311 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:01.910032 1311 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } }
I20260809 02:19:01.910154 1311 leader_election.cc:302] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d088ff0b8f740878af02f3bedac77b4; no voters:
I20260809 02:19:01.910341 1311 leader_election.cc:288] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260809 02:19:01.910422 1321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:01.910600 1321 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [term 1 LEADER]: Becoming Leader. State: Replica: 7d088ff0b8f740878af02f3bedac77b4, State: Running, Role: LEADER
I20260809 02:19:01.910624 1311 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:01.910712 1321 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } }
I20260809 02:19:01.911149 1322 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d088ff0b8f740878af02f3bedac77b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } } }
I20260809 02:19:01.911142 1323 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d088ff0b8f740878af02f3bedac77b4. Latest consensus state: current_term: 1 leader_uuid: "7d088ff0b8f740878af02f3bedac77b4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d088ff0b8f740878af02f3bedac77b4" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 42807 } } }
I20260809 02:19:01.911259 1322 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:01.911268 1323 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:01.911469 1327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:01.911967 1327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:01.913689 1327 catalog_manager.cc:1347] Generated new cluster ID: aafb504508b948248ed93e04b9104138
I20260809 02:19:01.913753 1327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:01.937855 1327 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:01.938081 1327 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:01.940163 1327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d088ff0b8f740878af02f3bedac77b4: Generated new TSK 0
I20260809 02:19:01.940356 1327 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:01.945760 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1237
2026-08-09T02:19:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1443 ms)
[----------] 18 tests from AuthTokenIssuingTest (25313 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:02 dist-test-slave-qg60 krb5kdc[1347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:02 dist-test-slave-qg60 krb5kdc[1347](info): set up 2 sockets
Aug 09 02:19:02 dist-test-slave-qg60 krb5kdc[1347](info): commencing operation
krb5kdc: starting...
W20260809 02:19:03.038746 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:19:03.172497 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38303
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:03.256633 1363 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:03.256901 1363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:03.256919 1363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:03.256942 1363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:03.258139 1363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:03.258186 1363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:03.258200 1363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:03.258213 1363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:03.258226 1363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:03.259788 1363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:38303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:03.260044 1363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:03.260321 1363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:03.265913 1370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.265923 1372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.265935 1369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:03.266225 1363 server_base.cc:1034] running on GCE node
I20260809 02:19:03.266498 1363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:03.266821 1363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:03.267995 1363 hybrid_clock.cc:648] HybridClock initialized: now 1786241943267985 us; error 56 us; skew 500 ppm
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:03.270751 1363 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:19:03.271284 1363 webserver.cc:460] Webserver started at http://127.9.40.126:39461/ using document root <none> and password file <none>
I20260809 02:19:03.271544 1363 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:03.271618 1363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:03.271775 1363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:03.272986 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "bc04b345c36d4389bdc3a2cc9f84c849"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "b237928b7f9a1d38926131ed0f24fd5a"
server_key_iv: "6e0d2a4f643a3ab6d28ebf296dbce025"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.273396 1363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "bc04b345c36d4389bdc3a2cc9f84c849"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "b237928b7f9a1d38926131ed0f24fd5a"
server_key_iv: "6e0d2a4f643a3ab6d28ebf296dbce025"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.274758 1363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260809 02:19:03.275527 1379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:03.275705 1363 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:19:03.275808 1363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "bc04b345c36d4389bdc3a2cc9f84c849"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "b237928b7f9a1d38926131ed0f24fd5a"
server_key_iv: "6e0d2a4f643a3ab6d28ebf296dbce025"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.275899 1363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:03.296461 1363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:03.299109 1363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:03.299286 1363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:03.303784 1363 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:38303
I20260809 02:19:03.303825 1436 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:38303 every 8 connection(s)
I20260809 02:19:03.304224 1363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:03.305271 1437 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:03.306792 1437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.308547 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1363
I20260809 02:19:03.308677 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
I20260809 02:19:03.308943 9377 external_mini_cluster.cc:1428] Setting key 981db8a155b03712b84b1bc7250ed770
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
I20260809 02:19:03.318106 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.307582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60316 (local address 127.9.40.126:38303)
0809 02:19:03.307749 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.307753 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.307789 (+ 36us) server_negotiation.cc:407] Connection header received
0809 02:19:03.308305 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.308307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.308324 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.308413 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.309308 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.310213 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.311328 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.311835 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.311949 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.314443 (+ 2494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.314451 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.314452 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.314461 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.316410 (+ 1949us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.316844 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.316849 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.316849 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.316932 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.317202 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.317202 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.317203 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.317428 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.317523 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.317721 (+ 198us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.317856 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260809 02:19:03.318107 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.307470 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38303 (local address 127.0.0.1:60316)
0809 02:19:03.307648 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.307691 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.308162 (+ 471us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.308458 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.308461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.308882 (+ 421us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.309191 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.309200 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.310323 (+ 1123us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.310324 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.311236 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.311238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.311943 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.311944 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.312001 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.312587 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.312596 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.314348 (+ 1752us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.316501 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.316508 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.316509 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.316708 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.317004 (+ 296us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.317006 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.317006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.317094 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.317501 (+ 407us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.317506 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.317578 (+ 72us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.317672 (+ 94us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.317920 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"mutex_wait_us":100,"thread_start_us":93,"threads_started":1}
I20260809 02:19:03.318915 1437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.319360 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.319216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52426)
0809 02:19:03.319287 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.319334 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260809 02:19:03.319501 1382 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
I20260809 02:19:03.319932 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.319846 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52434)
0809 02:19:03.319889 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.319914 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 02:19:03.320046 1437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44825: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260809 02:19:03.337991 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:44825
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:03.369311 1437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } attempt: 1
I20260809 02:19:03.369871 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.369700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52448)
0809 02:19:03.369776 (+ 76us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.369829 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260809 02:19:03.370327 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.370222 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52454)
0809 02:19:03.370272 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.370304 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:19:03.370466 1437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44825: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
W20260809 02:19:03.424098 1444 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:03.424319 1444 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:03.424336 1444 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:03.424357 1444 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:03.425554 1444 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:03.425601 1444 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:03.425616 1444 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:03.425628 1444 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:03.425642 1444 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:03.427184 1444 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:44825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1444
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:03.427441 1444 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:03.427683 1444 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:03.432883 1451 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.432937 1453 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.432883 1450 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:03.433223 1444 server_base.cc:1034] running on GCE node
I20260809 02:19:03.433514 1444 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:03.433820 1444 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:03.434983 1444 hybrid_clock.cc:648] HybridClock initialized: now 1786241943434972 us; error 28 us; skew 500 ppm
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:03.437667 1444 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260809 02:19:03.438171 1444 webserver.cc:460] Webserver started at http://127.9.40.125:46669/ using document root <none> and password file <none>
I20260809 02:19:03.438426 1444 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:03.438493 1444 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:03.438645 1444 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:03.438643 1437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } attempt: 2
I20260809 02:19:03.439072 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.438950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52464)
0809 02:19:03.439010 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.439045 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:19:03.439412 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.439321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52476)
0809 02:19:03.439362 (+ 41us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.439389 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260809 02:19:03.439500 1437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:44825: Network error: Client connection negotiation failed: client connection to 127.9.40.125:44825: connect: Connection refused (error 111)
I20260809 02:19:03.439978 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data/instance:
uuid: "c2de1a89fc6a45bd870714418951bfb0"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "1c48800c092faa84c956bda172fb39ab"
server_key_iv: "de5271faa7f3d75868226e03018fcc52"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.440361 1444 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal/instance:
uuid: "c2de1a89fc6a45bd870714418951bfb0"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "1c48800c092faa84c956bda172fb39ab"
server_key_iv: "de5271faa7f3d75868226e03018fcc52"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.441816 1444 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260809 02:19:03.442606 1461 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:03.442770 1444 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:19:03.442874 1444 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "c2de1a89fc6a45bd870714418951bfb0"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "1c48800c092faa84c956bda172fb39ab"
server_key_iv: "de5271faa7f3d75868226e03018fcc52"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.442950 1444 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:03.451922 1444 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:03.454162 1444 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:03.454335 1444 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:03.458669 1444 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:44825
I20260809 02:19:03.458704 1521 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:44825 every 8 connection(s)
I20260809 02:19:03.459056 1444 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
I20260809 02:19:03.460062 1522 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:03.461792 1522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.463991 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1444
I20260809 02:19:03.464083 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-1/wal/instance
I20260809 02:19:03.464310 9377 external_mini_cluster.cc:1428] Setting key 3662aa26230580aee37c978b58d11381
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:19:03.471824 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.462603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37286 (local address 127.9.40.126:38303)
0809 02:19:03.462701 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.462703 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.462909 (+ 206us) server_negotiation.cc:407] Connection header received
0809 02:19:03.463377 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.463377 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.463389 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.463454 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.464183 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.465056 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.466099 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.466454 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.466535 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.468860 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.468862 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.468863 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.468874 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.470409 (+ 1535us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.470836 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.470838 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.470838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.470900 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.471146 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.471147 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.471147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.471269 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.471338 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.471632 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.471721 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260809 02:19:03.472033 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.462598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38303 (local address 127.0.0.1:37286)
0809 02:19:03.462776 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.462819 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.463292 (+ 473us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.463525 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.463528 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.463825 (+ 297us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.464025 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.464029 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.465177 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.465178 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.466001 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.466003 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.466569 (+ 566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.466570 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.466699 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.467285 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.467293 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.468686 (+ 1393us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.470501 (+ 1815us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.470514 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.470517 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.470757 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.470967 (+ 210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.470967 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.470968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.471078 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.471341 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.471349 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.471485 (+ 136us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.471608 (+ 123us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.471786 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":90,"threads_started":1}
I20260809 02:19:03.472720 1522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:03.480950 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.473034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52480)
0809 02:19:03.473095 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.473109 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.473160 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.473421 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.473423 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.473584 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.473646 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.473648 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.474516 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.474517 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.475129 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.475131 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.475583 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.475584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.475640 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.476196 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.476200 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.477641 (+ 1441us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.479700 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.479702 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.479702 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.479958 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.480173 (+ 215us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.480174 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.480175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.480244 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.480669 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.480670 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.480707 (+ 37us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.480808 (+ 101us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.480883 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:19:03.481192 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.473072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52480 (local address 127.9.40.125:44825)
0809 02:19:03.473267 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.473272 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.473290 (+ 18us) server_negotiation.cc:407] Connection header received
0809 02:19:03.473316 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.473318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.473329 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.473391 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.473721 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.474441 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.475205 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.475509 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.475566 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.477724 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.477732 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.477733 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.477740 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.479603 (+ 1863us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.480043 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.480046 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.480047 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.480102 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.480368 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.480369 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.480370 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.480592 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.480692 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.480857 (+ 165us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.481039 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":130,"threads_started":1}
I20260809 02:19:03.481870 1522 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.482245 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.482129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51238)
0809 02:19:03.482175 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.482226 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 02:19:03.482407 1465 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
I20260809 02:19:03.482897 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.482810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51254)
0809 02:19:03.482857 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.482881 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:19:03.483026 1522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46807: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260809 02:19:03.497427 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46807
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:41975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:03.530648 1437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } attempt: 3
I20260809 02:19:03.532274 1522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } attempt: 1
I20260809 02:19:03.532694 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.532549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51262)
0809 02:19:03.532609 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.532654 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260809 02:19:03.533161 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.533051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51278)
0809 02:19:03.533101 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.533134 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:19:03.533301 1522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46807: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:03.538502 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.531093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52494)
0809 02:19:03.531150 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.531167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.531235 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.531394 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.531395 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.531573 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.531662 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.531664 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.532482 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.532483 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.533282 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.533285 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.533825 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.533826 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.533896 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.534370 (+ 474us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.534380 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.535601 (+ 1221us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.537498 (+ 1897us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.537499 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.537499 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.537667 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.537900 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.537900 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.537901 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.537954 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.538263 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.538264 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.538305 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.538359 (+ 54us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.538436 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260809 02:19:03.538582 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.531172 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52494 (local address 127.9.40.125:44825)
0809 02:19:03.531247 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.531250 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.531261 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:19:03.531305 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.531306 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.531323 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.531385 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.531752 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.532368 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.533383 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.533736 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.533818 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.535684 (+ 1866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.535686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.535686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.535693 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.537409 (+ 1716us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.537746 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.537748 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.537748 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.537822 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.538021 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.538022 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.538022 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.538188 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.538270 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.538404 (+ 134us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.538485 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260809 02:19:03.538939 1437 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.539327 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.539207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51280)
0809 02:19:03.539260 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.539297 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:19:03.539677 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.539583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51292)
0809 02:19:03.539630 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.539658 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260809 02:19:03.539775 1437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46807: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
I20260809 02:19:03.570948 1437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } attempt: 1
I20260809 02:19:03.571530 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.571359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51308)
0809 02:19:03.571434 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.571492 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
I20260809 02:19:03.571977 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.571864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51322)
0809 02:19:03.571919 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.571950 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260809 02:19:03.572111 1437 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46807: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
W20260809 02:19:03.586887 1531 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:03.587112 1531 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:03.587131 1531 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:03.587150 1531 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:03.588303 1531 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:03.588351 1531 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:03.588364 1531 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:03.588400 1531 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:03.588429 1531 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:03.590041 1531 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:41975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:38303,127.9.40.125:44825,127.9.40.124:46807
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:46807
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1531
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:03.590317 1531 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:03.590572 1531 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:03.595698 1537 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.595727 1538 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:03.595727 1540 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:03.595932 1531 server_base.cc:1034] running on GCE node
I20260809 02:19:03.596292 1531 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:03.596611 1531 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:03.597786 1531 hybrid_clock.cc:648] HybridClock initialized: now 1786241943597775 us; error 42 us; skew 500 ppm
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:03.600374 1531 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260809 02:19:03.600943 1531 webserver.cc:460] Webserver started at http://127.9.40.124:41505/ using document root <none> and password file <none>
I20260809 02:19:03.601263 1531 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:03.601336 1531 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:03.601504 1531 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:03.601460 1522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } attempt: 2
I20260809 02:19:03.601914 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.601785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51324)
0809 02:19:03.601847 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.601885 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260809 02:19:03.602252 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.602150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51340)
0809 02:19:03.602197 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.602231 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260809 02:19:03.602344 1522 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:46807: Network error: Client connection negotiation failed: client connection to 127.9.40.124:46807: connect: Connection refused (error 111)
I20260809 02:19:03.602877 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data/instance:
uuid: "8c0166909f1740ebae03aa77bf1744ec"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "bca4f3c662e23090bb596ba8e6df5d03"
server_key_iv: "5551b489bf1b857ed7f6882793e945dc"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.603300 1531 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal/instance:
uuid: "8c0166909f1740ebae03aa77bf1744ec"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "bca4f3c662e23090bb596ba8e6df5d03"
server_key_iv: "5551b489bf1b857ed7f6882793e945dc"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.604707 1531 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260809 02:19:03.605625 1549 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:03.605786 1531 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260809 02:19:03.605873 1531 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "8c0166909f1740ebae03aa77bf1744ec"
format_stamp: "Formatted at 2026-08-09 02:19:03 on dist-test-slave-qg60"
server_key: "bca4f3c662e23090bb596ba8e6df5d03"
server_key_iv: "5551b489bf1b857ed7f6882793e945dc"
server_key_version: "encryptionkey@0"
I20260809 02:19:03.605930 1531 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:03.620482 1531 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:03.622813 1531 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:03.622932 1531 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:03.626987 1531 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:46807
I20260809 02:19:03.627048 1603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:46807 every 8 connection(s)
I20260809 02:19:03.627388 1531 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
I20260809 02:19:03.628441 1604 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:03.630142 1604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.633677 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1531
I20260809 02:19:03.633776 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786241669736972-9377-0/minicluster-data/master-2/wal/instance
I20260809 02:19:03.633975 9377 external_mini_cluster.cc:1428] Setting key 968ed9ec48c81aba91734182ccf57729
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:03.641193 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.631005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37302 (local address 127.9.40.126:38303)
0809 02:19:03.631093 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.631096 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.631287 (+ 191us) server_negotiation.cc:407] Connection header received
0809 02:19:03.631851 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.631852 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.631865 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.631913 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.632563 (+ 650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.633105 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.634447 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.634747 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.634812 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.637274 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.637276 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.637276 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.637284 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.639146 (+ 1862us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.640091 (+ 945us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.640092 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.640093 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.640156 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.640437 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.640438 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.640439 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.640571 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.640623 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.640999 (+ 376us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.641093 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260809 02:19:03.641512 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.630976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:38303 (local address 127.0.0.1:37302)
0809 02:19:03.631162 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.631201 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.631727 (+ 526us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.632002 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.632005 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.632266 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.632466 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.632470 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.633215 (+ 745us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.633216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.634372 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.634375 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.634840 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.634842 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.634961 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.635546 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.635554 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.637191 (+ 1637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.639247 (+ 2056us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.639254 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.639255 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.639480 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.640239 (+ 759us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.640239 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.640240 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.640361 (+ 121us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.640634 (+ 273us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.640640 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.640839 (+ 199us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.640961 (+ 122us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.641237 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":90,"threads_started":1}
I20260809 02:19:03.642148 1604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:03.643827 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.634458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51346 (local address 127.9.40.124:46807)
0809 02:19:03.634721 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.634725 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.634759 (+ 34us) server_negotiation.cc:407] Connection header received
0809 02:19:03.634778 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.634780 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.634798 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.634889 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.635736 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.636592 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.637536 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.637940 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.637996 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.640480 (+ 2484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.640486 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.640486 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.640492 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.642639 (+ 2147us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.643018 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.643021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.643021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.643087 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.643265 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.643266 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.643267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.643437 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.643614 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.643656 (+ 42us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.643730 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":136,"threads_started":1}
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:03.647435 1437 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } attempt: 2
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:19:03.650200 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.642426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:44825 (local address 127.0.0.1:52506)
0809 02:19:03.642547 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.642556 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.642592 (+ 36us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.642722 (+ 130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.642723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.642881 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.642923 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.642924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.643888 (+ 964us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.643889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.644697 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.644699 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.645289 (+ 590us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.645290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.645373 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.645752 (+ 379us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.645757 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.647125 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.649087 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.649088 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.649089 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.649259 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.649543 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.649544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.649544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.649601 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.649870 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.649871 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.649914 (+ 43us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.649986 (+ 72us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.650074 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100}
I20260809 02:19:03.650238 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.642497 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52506 (local address 127.9.40.125:44825)
0809 02:19:03.642569 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.642571 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.642586 (+ 15us) server_negotiation.cc:407] Connection header received
0809 02:19:03.642639 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.642640 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.642652 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.642709 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.643085 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.643799 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.644805 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.645190 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.645264 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.647233 (+ 1969us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.647235 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.647235 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.647242 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.648988 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.649356 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.649357 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.649357 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.649413 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.649677 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.649678 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.649678 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.649795 (+ 117us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.649865 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.650028 (+ 163us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.650111 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260809 02:19:03.650846 1604 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:03.653260 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.644623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37314 (local address 127.9.40.126:38303)
0809 02:19:03.644662 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.644664 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.644673 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:03.644694 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.644694 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.644703 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.644790 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.645352 (+ 562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.646118 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.646907 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.647117 (+ 210us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.647193 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.649943 (+ 2750us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.649944 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.649945 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.649952 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.651998 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.652392 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.652393 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.652394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.652449 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.652670 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.652671 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.652671 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.652826 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.653012 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.653050 (+ 38us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.653142 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:03.656213 1439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.647885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51354)
0809 02:19:03.647933 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.647942 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.647981 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.648259 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.648260 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.648406 (+ 146us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.648437 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.648439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.649320 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.649322 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.650121 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.650123 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.651119 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.651120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.651192 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.651530 (+ 338us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.651535 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.653127 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.655077 (+ 1950us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.655077 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.655078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.655299 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.655512 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.655513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.655513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.655580 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.655892 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.655893 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.655940 (+ 47us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.656011 (+ 71us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.656110 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33}
I20260809 02:19:03.656508 1608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.647930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51354 (local address 127.9.40.124:46807)
0809 02:19:03.647968 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.647970 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.647979 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:03.648134 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.648136 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.648151 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.648252 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.648524 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.649217 (+ 693us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.650470 (+ 1253us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.650744 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.651101 (+ 357us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.653214 (+ 2113us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.653215 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.653216 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.653222 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.654954 (+ 1732us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.655380 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.655382 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.655382 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.655436 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.655654 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.655654 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.655655 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.655792 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.655893 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.656338 (+ 445us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.656416 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.658798 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: Bootstrap starting.
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:03.659638 1437 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:03.660003 1437 log.cc:824] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:03.660189 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.651196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.9.40.124:46807)
0809 02:19:03.651688 (+ 492us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.651690 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.651699 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:03.651718 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.651718 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.651729 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.651806 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.652100 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.652882 (+ 782us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.653800 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.654163 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.654226 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.656492 (+ 2266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.656493 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.656494 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.656501 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.658823 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.659238 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.659239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.659239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.659306 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.659500 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.659502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.659502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.659735 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.659804 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.659930 (+ 126us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.660042 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":127,"threads_started":1}
I20260809 02:19:03.660637 1606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.651118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51362)
0809 02:19:03.651345 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.651355 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.651395 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.651830 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.651831 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.651985 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.652022 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.652023 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.652963 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.652964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.653699 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.653701 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.654272 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.654273 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.654328 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.654677 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.654687 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.656181 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.658972 (+ 2791us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.658973 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.658974 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.659151 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.659374 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.659375 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.659375 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.659430 (+ 55us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.659795 (+ 365us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.659796 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.659836 (+ 40us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.659906 (+ 70us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.660153 (+ 247us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":208,"mutex_wait_us":91}
I20260809 02:19:03.662231 1604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: Bootstrap starting.
I20260809 02:19:03.662261 1437 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: No bootstrap required, opened a new log
I20260809 02:19:03.662946 1604 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:03.663138 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.654194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52512 (local address 127.9.40.125:44825)
0809 02:19:03.654254 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.654256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.654323 (+ 67us) server_negotiation.cc:407] Connection header received
0809 02:19:03.654398 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.654399 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.654408 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.654460 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.655033 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.655753 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.656767 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.657083 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.657132 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.660040 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.660041 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.660042 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.660049 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.661791 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.662216 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.662217 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.662217 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.662279 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.662546 (+ 267us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.662547 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.662547 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.662666 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.662841 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.662878 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.662958 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260809 02:19:03.663265 1604 log.cc:824] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:03.664155 1604 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: No bootstrap required, opened a new log
I20260809 02:19:03.664541 1437 raft_consensus.cc:348] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.664705 1437 raft_consensus.cc:729] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc04b345c36d4389bdc3a2cc9f84c849, State: Initialized, Role: FOLLOWER
I20260809 02:19:03.664815 1437 consensus_queue.cc:260] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.665163 1437 sys_catalog.cc:564] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:03.665764 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:19:03.666589 1604 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.666766 1604 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c0166909f1740ebae03aa77bf1744ec, State: Initialized, Role: FOLLOWER
I20260809 02:19:03.666898 1604 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.666924 1407 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "bc04b345c36d4389bdc3a2cc9f84c849"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:37314
I20260809 02:19:03.667186 1407 raft_consensus.cc:482] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:19:03.667245 1407 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:03.667301 1604 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: configured and running, proceeding with master startup.
W20260809 02:19:03.667330 1626 catalog_manager.cc:1558] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:03.667371 1626 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:03.668056 1407 raft_consensus.cc:504] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.668256 1407 leader_election.cc:288] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [CANDIDATE]: Term 1 election: Requested vote from peers c2de1a89fc6a45bd870714418951bfb0 (127.9.40.125:44825), 8c0166909f1740ebae03aa77bf1744ec (127.9.40.124:46807)
I20260809 02:19:03.669034 1578 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "8c0166909f1740ebae03aa77bf1744ec"
I20260809 02:19:03.669152 1578 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [term 0 FOLLOWER]: Advancing to term 1
W20260809 02:19:03.669359 1642 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:03.669412 1642 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:03.670112 1613 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.670207 1613 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.670608 1578 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc04b345c36d4389bdc3a2cc9f84c849 in term 1.
I20260809 02:19:03.670878 1628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.670979 1628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.671133 1382 leader_election.cc:302] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8c0166909f1740ebae03aa77bf1744ec, bc04b345c36d4389bdc3a2cc9f84c849; no voters:
I20260809 02:19:03.672166 1613 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:03.672417 1613 raft_consensus.cc:686] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [term 1 LEADER]: Becoming Leader. State: Replica: bc04b345c36d4389bdc3a2cc9f84c849, State: Running, Role: LEADER
I20260809 02:19:03.672544 1613 consensus_queue.cc:237] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.673260 1613 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc04b345c36d4389bdc3a2cc9f84c849. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.673381 1613 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:03.673585 1363 master_runner.cc:186] Error getting master registration for 127.9.40.125:44825: OK, instance_id { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" instance_seqno: 1786241943457472 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:19:03.675937 1363 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:19:03.676167 1531 master_runner.cc:186] Error getting master registration for 127.9.40.125:44825: OK, instance_id { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" instance_seqno: 1786241943457472 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:19:03.678931 1531 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:19:03.679646 1654 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:03.679862 1654 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:03.681244 1440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.669736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37324 (local address 127.9.40.126:38303)
0809 02:19:03.670857 (+ 1121us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.670860 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.670870 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:19:03.670893 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.670894 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.670904 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.670938 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.671981 (+ 1043us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.672713 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.673837 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.674217 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.674265 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.675923 (+ 1658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.675925 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.675925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.675933 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.677683 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.678328 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.678329 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.678329 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.678392 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.678725 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.678725 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.678726 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.678857 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.678943 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.681049 (+ 2106us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.681143 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1090}
I20260809 02:19:03.686372 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.670299 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51370 (local address 127.9.40.124:46807)
0809 02:19:03.670339 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.670341 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.671692 (+ 1351us) server_negotiation.cc:407] Connection header received
0809 02:19:03.671792 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.671793 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.671806 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.671835 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.674331 (+ 2496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.675078 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.677974 (+ 2896us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.678282 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.678328 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.680282 (+ 1954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.680283 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.680284 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.680291 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.683668 (+ 3377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.685212 (+ 1544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.685214 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.685226 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.685284 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.685598 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.685599 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.685611 (+ 12us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.685745 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.685845 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.686181 (+ 336us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.686272 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.686920 1578 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [term 1 FOLLOWER]: Refusing update from remote peer bc04b345c36d4389bdc3a2cc9f84c849: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:03.687635 1652 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:03.688177 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.670012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52528 (local address 127.9.40.125:44825)
0809 02:19:03.670056 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.670058 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.670610 (+ 552us) server_negotiation.cc:407] Connection header received
0809 02:19:03.670732 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.670734 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.670746 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.670776 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.672554 (+ 1778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.673305 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.676060 (+ 2755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.676343 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.676388 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.681301 (+ 4913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.681303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.681303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.681312 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.684575 (+ 3263us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.687038 (+ 2463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.687040 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.687040 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.687125 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.687423 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.687424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.687424 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.687576 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.687653 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.687970 (+ 317us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.688075 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.689545 1628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc04b345c36d4389bdc3a2cc9f84c849. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.689651 1628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.691092 1613 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.691244 1613 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:03.691344 1654 catalog_manager.cc:1347] Generated new cluster ID: dffe117a2add4fb09b6071002b8c6f6a
I20260809 02:19:03.691380 1654 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:03.693099 1657 mvcc.cc:205] Tried to move back new op lower bound from 7316447001338216448 to 7316447001284300800. Current Snapshot: MvccSnapshot[applied={T|T < 7316447001338216448 or (T in {7316447001338216448})}]
I20260809 02:19:03.693460 1522 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } attempt: 3
I20260809 02:19:03.700623 1628 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.700872 1628 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec [sys.catalog]: This master's current role is: FOLLOWER
Aug 09 02:19:03 dist-test-slave-qg60 krb5kdc[1347](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241943, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:03.706995 1524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.693731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:46807 (local address 127.0.0.1:51382)
0809 02:19:03.693787 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:03.693799 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:03.693850 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:03.697377 (+ 3527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:03.697378 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:03.697573 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:03.697627 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.697629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.698877 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.698878 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.699705 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:03.699707 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.700959 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.700961 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:03.701026 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.701555 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:03.701561 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:03.703823 (+ 2262us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:03.705809 (+ 1986us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.705810 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.705811 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.706051 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.706263 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:03.706264 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:03.706264 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:03.706321 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:03.706633 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:03.706634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:03.706680 (+ 46us) client_negotiation.cc:769] Sending connection context
0809 02:19:03.706743 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:19:03.706881 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22}
I20260809 02:19:03.707365 1609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.697117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51382 (local address 127.9.40.124:46807)
0809 02:19:03.697252 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.697254 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.697262 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:19:03.697283 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.697284 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.697294 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.697354 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.697727 (+ 373us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.698487 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.699808 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.700120 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.700183 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.703902 (+ 3719us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:03.703903 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:03.703904 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:03.703912 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:03.705707 (+ 1795us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.706135 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.706137 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.706137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.706196 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:03.706383 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:03.706384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:03.706384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:03.706535 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:03.706605 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.707179 (+ 574us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.707266 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113}
I20260809 02:19:03.709864 1522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: Bootstrap starting.
I20260809 02:19:03.710644 1522 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:03.710764 1654 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:03.710954 1522 log.cc:824] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:03.711463 1654 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:03.720326 1522 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: No bootstrap required, opened a new log
I20260809 02:19:03.721426 1654 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849: Generated new TSK 0
I20260809 02:19:03.721662 1654 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:03.722929 1522 raft_consensus.cc:348] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.723109 1522 raft_consensus.cc:729] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c2de1a89fc6a45bd870714418951bfb0, State: Initialized, Role: FOLLOWER
I20260809 02:19:03.723286 1522 consensus_queue.cc:260] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } }
I20260809 02:19:03.723708 1522 sys_catalog.cc:564] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:03.723790 1491 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "c2de1a89fc6a45bd870714418951bfb0"
I20260809 02:19:03.723908 1491 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:03.724859 1491 raft_consensus.cc:2454] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc04b345c36d4389bdc3a2cc9f84c849 in term 1.
I20260809 02:19:03.725278 1490 raft_consensus.cc:1261] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [term 1 FOLLOWER]: Refusing update from remote peer bc04b345c36d4389bdc3a2cc9f84c849: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:03.725615 1613 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:03.727382 1680 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.727541 1680 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.727648 1680 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bc04b345c36d4389bdc3a2cc9f84c849. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.727749 1680 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.727818 1680 sys_catalog.cc:455] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:03.727905 1680 sys_catalog.cc:458] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:03.729264 1678 mvcc.cc:205] Tried to move back new op lower bound from 7316447001476677632 to 7316447001284300800. Current Snapshot: MvccSnapshot[applied={T|T < 7316447001338216448}]
I20260809 02:19:03.734817 1675 catalog_manager.cc:1259] Loaded cluster ID: dffe117a2add4fb09b6071002b8c6f6a
I20260809 02:19:03.734890 1675 catalog_manager.cc:1552] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: loading cluster ID for follower catalog manager: success
I20260809 02:19:03.736188 1675 catalog_manager.cc:1574] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: acquiring CA information for follower catalog manager: success
I20260809 02:19:03.736543 1675 catalog_manager.cc:1602] T 00000000000000000000000000000000 P c2de1a89fc6a45bd870714418951bfb0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260809 02:19:03.751009 1440 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.749122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37336 (local address 127.9.40.126:38303)
0809 02:19:03.749174 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.749176 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.750209 (+ 1033us) server_negotiation.cc:407] Connection header received
0809 02:19:03.750336 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.750337 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.750351 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.750392 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.750921 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:37336: BlockingRecv error: recv got EOF from 127.0.0.1:37336 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:19:03.751868 1527 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.749527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52542 (local address 127.9.40.125:44825)
0809 02:19:03.749586 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.749588 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.751310 (+ 1722us) server_negotiation.cc:407] Connection header received
0809 02:19:03.751409 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.751410 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.751422 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.751452 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.751788 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52542: BlockingRecv error: recv got EOF from 127.0.0.1:52542 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260809 02:19:03.752794 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.749877 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51394 (local address 127.9.40.124:46807)
0809 02:19:03.749931 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.749933 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.752139 (+ 2206us) server_negotiation.cc:407] Connection header received
0809 02:19:03.752258 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.752259 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.752273 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.752308 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.752704 (+ 396us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51394: BlockingRecv error: recv got EOF from 127.0.0.1:51394 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:03.764981 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.761117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52558 (local address 127.9.40.125:44825)
0809 02:19:03.761163 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.761165 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.761448 (+ 283us) server_negotiation.cc:407] Connection header received
0809 02:19:03.761580 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.761581 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.761634 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.761682 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.761915 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.762720 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.763911 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.764228 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.764269 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.764507 (+ 238us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.764660 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.764714 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.764866 (+ 152us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.764889 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:19:03.768808 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.767762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51410 (local address 127.9.40.124:46807)
0809 02:19:03.767806 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.767808 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.768057 (+ 249us) server_negotiation.cc:407] Connection header received
0809 02:19:03.768177 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.768178 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.768200 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.768257 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.768747 (+ 490us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51410: BlockingRecv error: recv got EOF from 127.0.0.1:51410 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.774588 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.771134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52562 (local address 127.9.40.125:44825)
0809 02:19:03.771171 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.771173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.771427 (+ 254us) server_negotiation.cc:407] Connection header received
0809 02:19:03.771523 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.771524 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.771534 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.771576 (+ 42us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.771766 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.772505 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.773733 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.774055 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.774095 (+ 40us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.774290 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.774324 (+ 34us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.774357 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.774478 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.774499 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:19:03.777983 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.777153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51414 (local address 127.9.40.124:46807)
0809 02:19:03.777195 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.777197 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.777417 (+ 220us) server_negotiation.cc:407] Connection header received
0809 02:19:03.777514 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.777515 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.777526 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.777568 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.777924 (+ 356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51414: BlockingRecv error: recv got EOF from 127.0.0.1:51414 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.784446 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.781155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52564 (local address 127.9.40.125:44825)
0809 02:19:03.781204 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.781206 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.781454 (+ 248us) server_negotiation.cc:407] Connection header received
0809 02:19:03.781564 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.781565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.781576 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.781607 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.781785 (+ 178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.782532 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.783651 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.783949 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.783988 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.784180 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.784215 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.784241 (+ 26us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.784344 (+ 103us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.784364 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260809 02:19:03.787616 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.786811 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51418 (local address 127.9.40.124:46807)
0809 02:19:03.786851 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.786853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.787058 (+ 205us) server_negotiation.cc:407] Connection header received
0809 02:19:03.787158 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.787159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.787169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.787198 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.787546 (+ 348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51418: BlockingRecv error: recv got EOF from 127.0.0.1:51418 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:19:03.796249 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.792885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52566 (local address 127.9.40.125:44825)
0809 02:19:03.792927 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.792929 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.793195 (+ 266us) server_negotiation.cc:407] Connection header received
0809 02:19:03.793296 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.793297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.793307 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.793336 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.793508 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.794263 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.795386 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.795697 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.795738 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.795933 (+ 195us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.795975 (+ 42us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.796005 (+ 30us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.796146 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.796166 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:19:03.799467 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.798666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51430 (local address 127.9.40.124:46807)
0809 02:19:03.798708 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.798710 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.798927 (+ 217us) server_negotiation.cc:407] Connection header received
0809 02:19:03.799024 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.799025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.799036 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.799067 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.799407 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51430: BlockingRecv error: recv got EOF from 127.0.0.1:51430 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:03.812271 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.808830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52576 (local address 127.9.40.125:44825)
0809 02:19:03.808874 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.808876 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.809184 (+ 308us) server_negotiation.cc:407] Connection header received
0809 02:19:03.809292 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.809293 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.809304 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.809333 (+ 29us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.809514 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.810295 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.811467 (+ 1172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.811770 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.811809 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.812007 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.812038 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.812065 (+ 27us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.812167 (+ 102us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.812186 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:19:03.815663 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.814809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51446 (local address 127.9.40.124:46807)
0809 02:19:03.814855 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.814857 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.815078 (+ 221us) server_negotiation.cc:407] Connection header received
0809 02:19:03.815179 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.815180 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.815192 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.815222 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.815603 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51446: BlockingRecv error: recv got EOF from 127.0.0.1:51446 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
I20260809 02:19:03.836582 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.833134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52588 (local address 127.9.40.125:44825)
0809 02:19:03.833181 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.833183 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.833474 (+ 291us) server_negotiation.cc:407] Connection header received
0809 02:19:03.833602 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.833603 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.833614 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.833646 (+ 32us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.833844 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.834546 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.835730 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.836014 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.836063 (+ 49us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.836283 (+ 220us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.836319 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.836348 (+ 29us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.836463 (+ 115us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.836484 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260809 02:19:03.840066 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.839147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51458 (local address 127.9.40.124:46807)
0809 02:19:03.839192 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.839194 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.839452 (+ 258us) server_negotiation.cc:407] Connection header received
0809 02:19:03.839571 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.839573 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.839585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.839617 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.840006 (+ 389us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51458: BlockingRecv error: recv got EOF from 127.0.0.1:51458 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:19:03.877080 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.873518 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52598 (local address 127.9.40.125:44825)
0809 02:19:03.873577 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.873579 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.873921 (+ 342us) server_negotiation.cc:407] Connection header received
0809 02:19:03.874033 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.874035 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.874046 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.874077 (+ 31us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.874279 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.875015 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.876192 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.876460 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.876506 (+ 46us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.876782 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.876820 (+ 38us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.876852 (+ 32us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.876970 (+ 118us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.876997 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:19:03.880662 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.879724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51466 (local address 127.9.40.124:46807)
0809 02:19:03.879767 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.879769 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.880007 (+ 238us) server_negotiation.cc:407] Connection header received
0809 02:19:03.880110 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.880111 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.880123 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.880154 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.880601 (+ 447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51466: BlockingRecv error: recv got EOF from 127.0.0.1:51466 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:03.948415 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:03.945917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52612 (local address 127.9.40.125:44825)
0809 02:19:03.945980 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.945982 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.946251 (+ 269us) server_negotiation.cc:407] Connection header received
0809 02:19:03.946320 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.946321 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.946333 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.946376 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:03.946528 (+ 152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.947042 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.947788 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:03.947965 (+ 177us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:03.948016 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:03.948151 (+ 135us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:03.948178 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:03.948217 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:03.948281 (+ 64us) server_negotiation.cc:299] Negotiation successful
0809 02:19:03.948298 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
W20260809 02:19:03.950979 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:03.950373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51468 (local address 127.9.40.124:46807)
0809 02:19:03.950446 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:03.950448 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:03.950480 (+ 32us) server_negotiation.cc:407] Connection header received
0809 02:19:03.950559 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:03.950560 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:03.950570 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:03.950627 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:03.950931 (+ 304us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51468: BlockingRecv error: recv got EOF from 127.0.0.1:51468 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
I20260809 02:19:04.082672 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.080140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52624 (local address 127.9.40.125:44825)
0809 02:19:04.080244 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.080246 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.080297 (+ 51us) server_negotiation.cc:407] Connection header received
0809 02:19:04.080350 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.080351 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.080361 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.080417 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.080614 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.081153 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.081981 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.082159 (+ 178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.082209 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.082343 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.082370 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.082409 (+ 39us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.082530 (+ 121us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.082547 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
W20260809 02:19:04.085041 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:04.084374 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51478 (local address 127.9.40.124:46807)
0809 02:19:04.084423 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.084425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.084512 (+ 87us) server_negotiation.cc:407] Connection header received
0809 02:19:04.084551 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.084552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.084563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.084609 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:04.084988 (+ 379us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51478: BlockingRecv error: recv got EOF from 127.0.0.1:51478 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:04.134306 1661 sys_catalog.cc:455] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc04b345c36d4389bdc3a2cc9f84c849" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 38303 } } peers { permanent_uuid: "c2de1a89fc6a45bd870714418951bfb0" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 44825 } } peers { permanent_uuid: "8c0166909f1740ebae03aa77bf1744ec" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 46807 } } }
I20260809 02:19:04.134495 1661 sys_catalog.cc:458] T 00000000000000000000000000000000 P bc04b345c36d4389bdc3a2cc9f84c849 [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:04.344857 1527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.342404 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52632 (local address 127.9.40.125:44825)
0809 02:19:04.342494 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.342496 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.342505 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:04.342533 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.342534 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.342543 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.342596 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.342738 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.343270 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.344030 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.344210 (+ 180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.344273 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.344431 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.344459 (+ 28us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.344499 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.344643 (+ 144us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.344696 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260809 02:19:04.347711 1609 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:04.347075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51486 (local address 127.9.40.124:46807)
0809 02:19:04.347125 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.347127 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.347177 (+ 50us) server_negotiation.cc:407] Connection header received
0809 02:19:04.347256 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.347257 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.347266 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.347316 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:04.347659 (+ 343us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51486: BlockingRecv error: recv got EOF from 127.0.0.1:51486 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:04.669860 1642 catalog_manager.cc:1259] Loaded cluster ID: dffe117a2add4fb09b6071002b8c6f6a
I20260809 02:19:04.669963 1642 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: loading cluster ID for follower catalog manager: success
I20260809 02:19:04.670768 1642 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: acquiring CA information for follower catalog manager: success
I20260809 02:19:04.671078 1642 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 8c0166909f1740ebae03aa77bf1744ec: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:19:04.863621 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.861196 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52634 (local address 127.9.40.125:44825)
0809 02:19:04.861355 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.861357 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.861369 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:19:04.861397 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.861398 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.861409 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.861480 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.861614 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.862123 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.862841 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.863025 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.863075 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.863224 (+ 149us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.863289 (+ 65us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.863361 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.863473 (+ 112us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.863496 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":79,"threads_started":1}
I20260809 02:19:04.868405 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.865709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51494 (local address 127.9.40.124:46807)
0809 02:19:04.865833 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.865836 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.865846 (+ 10us) server_negotiation.cc:407] Connection header received
0809 02:19:04.865883 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.865884 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.865944 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.866036 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.866140 (+ 104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.866776 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.867552 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.867802 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.867861 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.868046 (+ 185us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.868174 (+ 128us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.868240 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.868277 (+ 37us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.868297 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":59,"threads_started":1}
I20260809 02:19:04.873070 1808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.870242 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52640 (local address 127.9.40.125:44825)
0809 02:19:04.870288 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.870290 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.870385 (+ 95us) server_negotiation.cc:407] Connection header received
0809 02:19:04.870493 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.870494 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.870507 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.870559 (+ 52us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.870726 (+ 167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.871468 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872289 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.872566 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872641 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.872796 (+ 155us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.872831 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.872887 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.872965 (+ 78us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.872992 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:04.873467 1816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.870350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51496 (local address 127.9.40.124:46807)
0809 02:19:04.870390 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.870392 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.870458 (+ 66us) server_negotiation.cc:407] Connection header received
0809 02:19:04.870551 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.870551 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.870560 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.870607 (+ 47us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.870783 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.871481 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872531 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.872910 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872967 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.873163 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.873202 (+ 39us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.873233 (+ 31us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.873374 (+ 141us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.873396 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:19:04.873683 1824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:04.870100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37350 (local address 127.9.40.126:38303)
0809 02:19:04.870258 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:04.870260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:04.870411 (+ 151us) server_negotiation.cc:407] Connection header received
0809 02:19:04.870549 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:04.870550 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:04.870618 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:04.870705 (+ 87us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:04.870826 (+ 121us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.871345 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872402 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:04.872712 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:04.872806 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:04.873071 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:04.873206 (+ 135us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:04.873270 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:04.873564 (+ 294us) server_negotiation.cc:299] Negotiation successful
0809 02:19:04.873598 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":93,"threads_started":1}
I20260809 02:19:04.875526 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1363
I20260809 02:19:04.883939 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1444
I20260809 02:19:04.891252 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1531
2026-08-09T02:19:04Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2946 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 09 02:19:04 dist-test-slave-qg60 krb5kdc[1828](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 09 02:19:04 dist-test-slave-qg60 krb5kdc[1828](info): set up 2 sockets
Aug 09 02:19:04 dist-test-slave-qg60 krb5kdc[1828](info): commencing operation
krb5kdc: starting...
W20260809 02:19:06.009678 9377 mini_kdc.cc:121] Time spent starting KDC: real 1.097s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/test-user.keytab.
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-09T02:19:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-09T02:19:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.9.40.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
Entry for principal kudu/127.9.40.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab.
I20260809 02:19:06.144109 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35217
--webserver_interface=127.9.40.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260809 02:19:06.244606 1844 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:06.245056 1844 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:06.245126 1844 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:06.245173 1844 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:06.246486 1844 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:06.246564 1844 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:06.246594 1844 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:06.246629 1844 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:06.246661 1844 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:06.248373 1844 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.126.keytab
--principal=kudu/127.9.40.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.126:35217
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.9.40.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1844
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:06.248690 1844 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:06.248997 1844 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:06.254686 1849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.254833 1844 server_base.cc:1034] running on GCE node
W20260809 02:19:06.254968 1850 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:06.255003 1852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.255332 1844 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:06.255656 1844 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:06.256804 1844 hybrid_clock.cc:648] HybridClock initialized: now 1786241946256775 us; error 36 us; skew 500 ppm
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:06.259439 1844 init.cc:377] Logged in from keytab as kudu/127.9.40.126@KRBTEST.COM (short username kudu)
I20260809 02:19:06.259968 1844 webserver.cc:460] Webserver started at http://127.9.40.126:42571/ using document root <none> and password file <none>
I20260809 02:19:06.260228 1844 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:06.260298 1844 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:06.260449 1844 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:06.261610 1844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data/instance:
uuid: "01e4b84dc4654708b711f6819346497f"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "ca77ba1441b7ffe01812a0901d8b11ad"
server_key_iv: "b0f7cf8f47f57f8768ed706da1e2eda1"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.261988 1844 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance:
uuid: "01e4b84dc4654708b711f6819346497f"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "ca77ba1441b7ffe01812a0901d8b11ad"
server_key_iv: "b0f7cf8f47f57f8768ed706da1e2eda1"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.263388 1844 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260809 02:19:06.264166 1859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:06.264343 1844 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260809 02:19:06.264436 1844 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal
uuid: "01e4b84dc4654708b711f6819346497f"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "ca77ba1441b7ffe01812a0901d8b11ad"
server_key_iv: "b0f7cf8f47f57f8768ed706da1e2eda1"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.264518 1844 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:06.274894 1844 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:06.276846 1844 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:06.277009 1844 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:06.281279 1844 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.126:35217
I20260809 02:19:06.281322 1917 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.126:35217 every 8 connection(s)
I20260809 02:19:06.281697 1844 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/data/info.pb
I20260809 02:19:06.282756 1918 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:06.284296 1918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:06.290058 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1844
I20260809 02:19:06.290198 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-0/wal/instance
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:19:06.290457 9377 external_mini_cluster.cc:1428] Setting key e05d903e6b9dd5ca32388aba37a13b87
I20260809 02:19:06.294629 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.285028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60220 (local address 127.9.40.126:35217)
0809 02:19:06.285248 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.285252 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.285283 (+ 31us) server_negotiation.cc:407] Connection header received
0809 02:19:06.285823 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.285825 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.285841 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.285920 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.286465 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.287276 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.288124 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.288426 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.288506 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.290926 (+ 2420us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.290934 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.290935 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.290941 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.292996 (+ 2055us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.293452 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.293455 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.293455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.293537 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.293813 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.293814 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.293815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.294030 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.294096 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.294329 (+ 233us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.294444 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":182,"thread_start_us":125,"threads_started":1}
I20260809 02:19:06.294742 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.284979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35217 (local address 127.0.0.1:60220)
0809 02:19:06.285163 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.285209 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.285688 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.285950 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.285952 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.286207 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.286369 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.286373 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.287359 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.287359 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.288035 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.288037 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.288524 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.288525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.288578 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.289221 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.289229 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.290840 (+ 1611us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.293088 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.293095 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.293097 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.293349 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.293610 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.293611 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.293611 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.293746 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.294103 (+ 357us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.294111 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.294193 (+ 82us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.294304 (+ 111us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.294534 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":64,"thread_start_us":85,"threads_started":1}
I20260809 02:19:06.295545 1918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:06.295928 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.295810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34092)
0809 02:19:06.295860 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.295908 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
W20260809 02:19:06.296111 1863 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
I20260809 02:19:06.296602 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.296505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34106)
0809 02:19:06.296554 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.296579 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:19:06.296756 1918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:42133: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.9.40.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
Entry for principal kudu/127.9.40.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab.
I20260809 02:19:06.319257 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:42133
--webserver_interface=127.9.40.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:06.346071 1918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } attempt: 1
I20260809 02:19:06.346611 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.346463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34118)
0809 02:19:06.346537 (+ 74us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.346579 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260809 02:19:06.346978 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.346880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34124)
0809 02:19:06.346924 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.346950 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260809 02:19:06.347067 1918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:42133: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
W20260809 02:19:06.404484 1928 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:06.404808 1928 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:06.404853 1928 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:06.404894 1928 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:06.406188 1928 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:06.406366 1928 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:06.406493 1928 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:06.406531 1928 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:06.406565 1928 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:06.408389 1928 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.125.keytab
--principal=kudu/127.9.40.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.125:42133
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.9.40.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1928
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:06.408715 1928 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:06.409000 1928 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:06.414337 1934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:06.414399 1936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.414485 1928 server_base.cc:1034] running on GCE node
W20260809 02:19:06.414407 1938 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.414779 1928 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:06.415143 1928 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:06.415220 1918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } attempt: 2
I20260809 02:19:06.415656 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.415517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34128)
0809 02:19:06.415581 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.415628 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 02:19:06.415985 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.415900 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34132)
0809 02:19:06.415944 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.415969 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260809 02:19:06.416098 1918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.125:42133: Network error: Client connection negotiation failed: client connection to 127.9.40.125:42133: connect: Connection refused (error 111)
I20260809 02:19:06.416320 1928 hybrid_clock.cc:648] HybridClock initialized: now 1786241946416300 us; error 39 us; skew 500 ppm
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:06.418938 1928 init.cc:377] Logged in from keytab as kudu/127.9.40.125@KRBTEST.COM (short username kudu)
I20260809 02:19:06.419574 1928 webserver.cc:460] Webserver started at http://127.9.40.125:34271/ using document root <none> and password file <none>
I20260809 02:19:06.419849 1928 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:06.419898 1928 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:06.420079 1928 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:06.421337 1928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data/instance:
uuid: "22376c134461482a9e998f27060e8fa8"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "6933d7af80ecaae177364c312d111434"
server_key_iv: "47209b4477069e89ef0a4b934f73a45a"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.421706 1928 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal/instance:
uuid: "22376c134461482a9e998f27060e8fa8"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "6933d7af80ecaae177364c312d111434"
server_key_iv: "47209b4477069e89ef0a4b934f73a45a"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.423395 1928 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260809 02:19:06.424250 1945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:06.424465 1928 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:19:06.424579 1928 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal
uuid: "22376c134461482a9e998f27060e8fa8"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "6933d7af80ecaae177364c312d111434"
server_key_iv: "47209b4477069e89ef0a4b934f73a45a"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.424659 1928 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:06.436650 1928 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:06.438758 1928 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:06.438951 1928 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:06.443445 1928 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.125:42133
I20260809 02:19:06.443480 2005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.125:42133 every 8 connection(s)
I20260809 02:19:06.443866 1928 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/data/info.pb
I20260809 02:19:06.445065 2006 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:06.445246 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 1928
I20260809 02:19:06.445340 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-1/wal/instance
I20260809 02:19:06.445582 9377 external_mini_cluster.cc:1428] Setting key 4319fd85aac680cb5d1c661b073b3e1e
I20260809 02:19:06.447691 2006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.9.40.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.9.40.124@KRBTEST.COM" created.
I20260809 02:19:06.457684 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.448524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60222 (local address 127.9.40.126:35217)
0809 02:19:06.448603 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.448605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.448842 (+ 237us) server_negotiation.cc:407] Connection header received
0809 02:19:06.449390 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.449392 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.449407 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.449459 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.450302 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.450928 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.451935 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.452166 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.452248 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.454722 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.454723 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.454724 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.454731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.456225 (+ 1494us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.456636 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.456637 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.456638 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.456733 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.456979 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.456980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.456980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.457092 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.457151 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.457441 (+ 290us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.457527 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260809 02:19:06.457827 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.448483 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35217 (local address 127.0.0.1:60222)
0809 02:19:06.448648 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.448691 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.449274 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.449544 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.449547 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.449800 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.450178 (+ 378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.450183 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.451043 (+ 860us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.451045 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.451833 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.451835 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.452287 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.452289 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.452401 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.453030 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.453038 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.454629 (+ 1591us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.456305 (+ 1676us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.456312 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.456314 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.456535 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.456812 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.456813 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.456813 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.456917 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.457151 (+ 234us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.457156 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.457274 (+ 118us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.457357 (+ 83us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.457531 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":78,"threads_started":1}
I20260809 02:19:06.458491 2006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:06.466602 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.458774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34140)
0809 02:19:06.458827 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.458839 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.458887 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.459176 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.459177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.459322 (+ 145us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.459369 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.459370 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.460226 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.460227 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.460779 (+ 552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.460781 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.461267 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.461267 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.461333 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.462034 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.462039 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.463314 (+ 1275us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.465491 (+ 2177us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.465492 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.465492 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.465669 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.465904 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.465905 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.465905 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.465958 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.466357 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.466357 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.466396 (+ 39us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.466446 (+ 50us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.466535 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:19:06.466734 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.458853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34140 (local address 127.9.40.125:42133)
0809 02:19:06.459024 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.459029 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.459050 (+ 21us) server_negotiation.cc:407] Connection header received
0809 02:19:06.459071 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.459073 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.459086 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.459177 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.459449 (+ 272us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.460124 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.460842 (+ 718us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.461172 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.461266 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.463408 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.463419 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.463420 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.463428 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.465389 (+ 1961us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.465758 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.465761 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.465761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.465827 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.466064 (+ 237us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.466065 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.466065 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.466279 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.466389 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.466488 (+ 99us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.466617 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":26,"server-negotiator.queue_time_us":137,"thread_start_us":84,"threads_started":1}
I20260809 02:19:06.467412 2006 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:06.468021 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.467853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58528)
0809 02:19:06.467934 (+ 81us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.467999 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260809 02:19:06.468201 1949 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
I20260809 02:19:06.468783 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.468624 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58530)
0809 02:19:06.468704 (+ 80us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.468757 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260809 02:19:06.468930 2006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39793: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
Entry for principal kudu/127.9.40.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab.
I20260809 02:19:06.476204 9377 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskQln13t/build/release/bin/kudu
/tmp/dist-test-taskQln13t/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:39793
--webserver_interface=127.9.40.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.9.40.84:35407
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260809 02:19:06.507264 1918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } attempt: 3
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:06.515275 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.507746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34150)
0809 02:19:06.507811 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.507827 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.507899 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.508083 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.508085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.508259 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.508327 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.508329 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.509163 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.509165 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.509698 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.509700 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.510127 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.510128 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.510212 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.510773 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.510779 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.512348 (+ 1569us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.514200 (+ 1852us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.514200 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.514201 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.514368 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.514619 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.514620 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.514620 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.514674 (+ 54us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.514969 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.514971 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.515025 (+ 54us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.515098 (+ 73us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.515201 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260809 02:19:06.515333 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.507820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34150 (local address 127.9.40.125:42133)
0809 02:19:06.507891 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.507894 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.507906 (+ 12us) server_negotiation.cc:407] Connection header received
0809 02:19:06.507971 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.507972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.507982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.508075 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.508414 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.509018 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.509801 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.510048 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.510129 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.512440 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.512442 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.512442 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.512450 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.514110 (+ 1660us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.514484 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.514485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.514486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.514549 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.514734 (+ 185us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.514734 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.514735 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.514871 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.514946 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.515143 (+ 197us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.515231 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260809 02:19:06.515777 1918 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:06.516136 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.516020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58542)
0809 02:19:06.516073 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.516110 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260809 02:19:06.516528 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.516407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58552)
0809 02:19:06.516463 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.516496 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260809 02:19:06.516640 1918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39793: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
I20260809 02:19:06.518152 2006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } attempt: 1
I20260809 02:19:06.518491 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.518384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58558)
0809 02:19:06.518436 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.518470 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260809 02:19:06.518836 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.518736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58564)
0809 02:19:06.518784 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.518811 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260809 02:19:06.518922 2006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39793: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
I20260809 02:19:06.547818 1918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } attempt: 1
I20260809 02:19:06.548404 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.548246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58574)
0809 02:19:06.548315 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.548367 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260809 02:19:06.548830 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.548703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58580)
0809 02:19:06.548773 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.548805 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260809 02:19:06.548943 1918 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39793: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
W20260809 02:19:06.566589 2014 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260809 02:19:06.566874 2014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260809 02:19:06.566910 2014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260809 02:19:06.566949 2014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260809 02:19:06.568272 2014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260809 02:19:06.568320 2014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260809 02:19:06.568336 2014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260809 02:19:06.568348 2014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260809 02:19:06.568360 2014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260809 02:19:06.570013 2014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.9.40.84:35407
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.9.40.126:35217,127.9.40.125:42133,127.9.40.124:39793
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/krb5kdc/kudu_127.9.40.124.keytab
--principal=kudu/127.9.40.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.9.40.124:39793
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.9.40.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 09 Aug 2026 01:32:21 UTC on ccycloud.quasar-japptc.root.comops.site
build id 4575764
I20260809 02:19:06.570228 2014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260809 02:19:06.570438 2014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260809 02:19:06.575632 2021 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260809 02:19:06.575619 2022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.575661 2014 server_base.cc:1034] running on GCE node
W20260809 02:19:06.575616 2024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260809 02:19:06.576190 2014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260809 02:19:06.576503 2014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260809 02:19:06.577682 2014 hybrid_clock.cc:648] HybridClock initialized: now 1786241946577651 us; error 53 us; skew 500 ppm
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260809 02:19:06.580417 2014 init.cc:377] Logged in from keytab as kudu/127.9.40.124@KRBTEST.COM (short username kudu)
I20260809 02:19:06.580917 2014 webserver.cc:460] Webserver started at http://127.9.40.124:35775/ using document root <none> and password file <none>
I20260809 02:19:06.581174 2014 fs_manager.cc:359] Metadata directory not provided
I20260809 02:19:06.581244 2014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260809 02:19:06.581401 2014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260809 02:19:06.582475 2014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data/instance:
uuid: "80a2f2bfa8ee43f1be816214617e2ace"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "5dc5b10510cdd7b3a7a73f4984f1b46c"
server_key_iv: "803745b3c9db85fcf78a696e8db3f1fa"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.582844 2014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal/instance:
uuid: "80a2f2bfa8ee43f1be816214617e2ace"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "5dc5b10510cdd7b3a7a73f4984f1b46c"
server_key_iv: "803745b3c9db85fcf78a696e8db3f1fa"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.584242 2014 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260809 02:19:06.585058 2031 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260809 02:19:06.585246 2014 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260809 02:19:06.585353 2014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data,/tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal
uuid: "80a2f2bfa8ee43f1be816214617e2ace"
format_stamp: "Formatted at 2026-08-09 02:19:06 on dist-test-slave-qg60"
server_key: "5dc5b10510cdd7b3a7a73f4984f1b46c"
server_key_iv: "803745b3c9db85fcf78a696e8db3f1fa"
server_key_version: "encryptionkey@0"
I20260809 02:19:06.585429 2014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260809 02:19:06.587055 2006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } attempt: 2
I20260809 02:19:06.587486 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.587361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58594)
0809 02:19:06.587421 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.587458 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260809 02:19:06.587860 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.587762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58610)
0809 02:19:06.587809 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.587841 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260809 02:19:06.587950 2006 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.9.40.124:39793: Network error: Client connection negotiation failed: client connection to 127.9.40.124:39793: connect: Connection refused (error 111)
I20260809 02:19:06.603276 2014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260809 02:19:06.605971 2014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260809 02:19:06.606153 2014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260809 02:19:06.610121 2014 rpc_server.cc:307] RPC server started. Bound to: 127.9.40.124:39793
I20260809 02:19:06.610193 2089 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.9.40.124:39793 every 8 connection(s)
I20260809 02:19:06.610572 2014 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/data/info.pb
I20260809 02:19:06.611691 2090 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260809 02:19:06.612442 9377 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskQln13t/build/release/bin/kudu as pid 2014
I20260809 02:19:06.612533 9377 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskQln13t/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786241669736972-9377-0/minicluster-data/master-2/wal/instance
I20260809 02:19:06.612857 9377 external_mini_cluster.cc:1428] Setting key 77ef9b2f3ae7fd998d8d1563aedb9e46
I20260809 02:19:06.613724 2090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:06.623963 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.614810 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60230 (local address 127.9.40.126:35217)
0809 02:19:06.614862 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.614864 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.615009 (+ 145us) server_negotiation.cc:407] Connection header received
0809 02:19:06.615070 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.615071 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.615080 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.615137 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.615657 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.616349 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.617710 (+ 1361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.617996 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.618077 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.620904 (+ 2827us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.620906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.620907 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.620915 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.622517 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.622980 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.622982 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.622982 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.623045 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.623315 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.623315 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.623316 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.623442 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.623513 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.623781 (+ 268us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.623864 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260809 02:19:06.624102 1918 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } attempt: 2
I20260809 02:19:06.624221 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.613412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.9.40.124:39793)
0809 02:19:06.613621 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.613629 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.613655 (+ 26us) server_negotiation.cc:407] Connection header received
0809 02:19:06.614074 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.614080 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.614137 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.614413 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.615243 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.616517 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.617613 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.618106 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.618193 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.621166 (+ 2973us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.621175 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.621175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.621189 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.622804 (+ 1615us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.623181 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.623184 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.623184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.623245 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.623450 (+ 205us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.623451 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.623451 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.623659 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.623821 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.623893 (+ 72us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.623975 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":27,"server-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260809 02:19:06.624456 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.614686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.126:35217 (local address 127.0.0.1:60230)
0809 02:19:06.614873 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.614923 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.615003 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.615171 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.615173 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.615455 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.615557 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.615562 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.616434 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.616434 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.617615 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.617618 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.618112 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.618113 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.618203 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.618855 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.618866 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.620810 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.622623 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.622631 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.622633 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.622880 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.623120 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.623121 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.623122 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.623238 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.623518 (+ 280us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.623526 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.623652 (+ 126us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.623747 (+ 95us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.623926 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":2,"thread_start_us":118,"threads_started":1}
I20260809 02:19:06.624962 2090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } has no permanent_uuid. Determining permanent_uuid...
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.126@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.126@KRBTEST.COM
I20260809 02:19:06.632954 1920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.624330 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58622)
0809 02:19:06.624376 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.624388 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.624433 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.624595 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.624597 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.624771 (+ 174us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.624823 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.624825 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.625760 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.625762 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.626578 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.626581 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.627080 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627081 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.627139 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.627675 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.627679 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.629317 (+ 1638us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.631235 (+ 1918us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.631236 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.631236 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.631957 (+ 721us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.632204 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632205 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.632205 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.632276 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.632571 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.632572 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.632625 (+ 53us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.632716 (+ 91us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.632856 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":24}
I20260809 02:19:06.633433 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.625291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34152 (local address 127.9.40.125:42133)
0809 02:19:06.625336 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.625338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.625381 (+ 43us) server_negotiation.cc:407] Connection header received
0809 02:19:06.625462 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.625463 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.625472 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.625528 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.625853 (+ 325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.626563 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627402 (+ 839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.627752 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627823 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.630570 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.630572 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.630572 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.630582 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.632171 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632524 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.632526 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.632526 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.632586 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632833 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.632835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.632835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.633014 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.633077 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.633256 (+ 179us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.633331 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260809 02:19:06.633428 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.625190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.125:42133 (local address 127.0.0.1:34152)
0809 02:19:06.625345 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.625353 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.625393 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.625531 (+ 138us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.625532 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.625670 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.625754 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.625756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.626663 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.626663 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.627287 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.627289 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.627878 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.627951 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.628300 (+ 349us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.628305 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.630162 (+ 1857us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.632243 (+ 2081us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632244 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.632244 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.632443 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.632647 (+ 204us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632648 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.632648 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.632709 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.633107 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.633108 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.633153 (+ 45us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.633216 (+ 63us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.633322 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140}
I20260809 02:19:06.633517 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.624422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58622 (local address 127.9.40.124:39793)
0809 02:19:06.624455 (+ 33us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.624457 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.624465 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:19:06.624484 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.624485 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.624493 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.624895 (+ 402us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.624927 (+ 32us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.625669 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.626661 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.626972 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627052 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.629407 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.629408 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.629408 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.629414 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.631137 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632049 (+ 912us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.632051 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.632051 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.632115 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.632351 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.632352 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.632352 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.632513 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.632575 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.633304 (+ 729us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.633392 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260809 02:19:06.634101 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.625177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60238 (local address 127.9.40.126:35217)
0809 02:19:06.625214 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.625216 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.625224 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:19:06.625285 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.625285 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.625294 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.625347 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.625941 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.626383 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627334 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.627574 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.627622 (+ 48us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.630699 (+ 3077us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.630700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.630701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.630708 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.632453 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.633028 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.633030 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.633030 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.633095 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.633337 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.633338 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.633338 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.633509 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.633696 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.633916 (+ 220us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.634003 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260809 02:19:06.634397 2090 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } has no permanent_uuid. Determining permanent_uuid...
I20260809 02:19:06.635514 1918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: Bootstrap starting.
I20260809 02:19:06.636134 1918 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:06.636448 1918 log.cc:824] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:06.637816 1918 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: No bootstrap required, opened a new log
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.124@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:06.640256 1918 raft_consensus.cc:348] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.640450 1918 raft_consensus.cc:729] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 01e4b84dc4654708b711f6819346497f, State: Initialized, Role: FOLLOWER
I20260809 02:19:06.640559 1918 consensus_queue.cc:260] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.641002 1918 sys_catalog.cc:564] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:06.640975 2096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.641072 2096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.642264 2094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.634672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58636)
0809 02:19:06.634726 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.634738 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.634787 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.634911 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.634912 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.635055 (+ 143us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.635085 (+ 30us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.635086 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.635820 (+ 734us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.635821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.636391 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.636393 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.636864 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.636865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.636916 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.637422 (+ 506us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.637428 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.638897 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.641052 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.641053 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.641054 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.641270 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.641454 (+ 184us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.641454 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.641455 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.641515 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.641877 (+ 362us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.641878 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.641919 (+ 41us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.642012 (+ 93us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.642138 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35,"mutex_wait_us":5}
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.9.40.125@KRBTEST.COM
I20260809 02:19:06.643467 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.634735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58636 (local address 127.9.40.124:39793)
0809 02:19:06.634766 (+ 31us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.634767 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.634775 (+ 8us) server_negotiation.cc:407] Connection header received
0809 02:19:06.634844 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.634845 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.634854 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.634908 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.635169 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.635740 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.636495 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.636805 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.636869 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.639004 (+ 2135us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.639005 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.639005 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.639013 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.640958 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.641346 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.641347 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.641347 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.641395 (+ 48us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.641584 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.641585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.641586 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.641804 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.641873 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.642033 (+ 160us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.643350 (+ 1317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1222,"server-negotiator.queue_time_us":16}
I20260809 02:19:06.645155 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: Bootstrap starting.
I20260809 02:19:06.646216 2090 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:06.646545 2090 log.cc:824] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:06.646642 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.637485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34162 (local address 127.9.40.125:42133)
0809 02:19:06.637524 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.637526 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.637535 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:06.637582 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.637583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.637593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.637643 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.638223 (+ 580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.638767 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.639625 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.639898 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.639974 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.643212 (+ 3238us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.643214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.643215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.643224 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.645013 (+ 1789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.645439 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.645440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.645441 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.645510 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.645755 (+ 245us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.645756 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.645756 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.645899 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.646041 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.646473 (+ 432us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.646555 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260809 02:19:06.647303 2090 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: No bootstrap required, opened a new log
I20260809 02:19:06.645043 1844 master_runner.cc:186] Error getting master registration for 127.9.40.125:42133: OK, instance_id { permanent_uuid: "22376c134461482a9e998f27060e8fa8" instance_seqno: 1786241946442204 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:19:06.647526 1844 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260809 02:19:06.643215 2107 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:06.647881 2107 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:06.649173 9377 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260809 02:19:06.649701 2090 raft_consensus.cc:348] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.649853 2090 raft_consensus.cc:729] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 80a2f2bfa8ee43f1be816214617e2ace, State: Initialized, Role: FOLLOWER
I20260809 02:19:06.649986 2090 consensus_queue.cc:260] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.650175 1889 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "01e4b84dc4654708b711f6819346497f"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60238
I20260809 02:19:06.650314 1889 raft_consensus.cc:482] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260809 02:19:06.650285 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.650360 1889 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:06.650365 2090 sys_catalog.cc:564] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:06.650374 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.651345 1889 raft_consensus.cc:504] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.651593 1889 leader_election.cc:288] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [CANDIDATE]: Term 1 election: Requested vote from peers 22376c134461482a9e998f27060e8fa8 (127.9.40.125:42133), 80a2f2bfa8ee43f1be816214617e2ace (127.9.40.124:39793)
W20260809 02:19:06.652293 2122 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:06.652347 2122 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:06.652032 2062 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e4b84dc4654708b711f6819346497f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "80a2f2bfa8ee43f1be816214617e2ace"
I20260809 02:19:06.652544 2062 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:06.653457 2062 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01e4b84dc4654708b711f6819346497f in term 1.
I20260809 02:19:06.653710 1864 leader_election.cc:302] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 01e4b84dc4654708b711f6819346497f, 80a2f2bfa8ee43f1be816214617e2ace; no voters:
I20260809 02:19:06.653839 2096 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 1 FOLLOWER]: Leader election won for term 1
I20260809 02:19:06.654068 2096 raft_consensus.cc:686] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [term 1 LEADER]: Becoming Leader. State: Replica: 01e4b84dc4654708b711f6819346497f, State: Running, Role: LEADER
I20260809 02:19:06.654189 2096 consensus_queue.cc:237] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.654446 2014 master_runner.cc:186] Error getting master registration for 127.9.40.125:42133: OK, instance_id { permanent_uuid: "22376c134461482a9e998f27060e8fa8" instance_seqno: 1786241946442204 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260809 02:19:06.656937 2014 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260809 02:19:06.657105 2129 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e4b84dc4654708b711f6819346497f. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.657233 2129 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:06.657529 2132 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260809 02:19:06.657840 2132 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260809 02:19:06.659641 2062 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [term 1 FOLLOWER]: Refusing update from remote peer 01e4b84dc4654708b711f6819346497f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:06.660034 2129 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [LEADER]: Connected to new peer: Peer: permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:06.661484 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e4b84dc4654708b711f6819346497f. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.661588 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.663266 2096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.663352 2096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: This master's current role is: LEADER
I20260809 02:19:06.663412 2133 mvcc.cc:205] Tried to move back new op lower bound from 7316447013514776576 to 7316447013496369152. Current Snapshot: MvccSnapshot[applied={T|T < 7316447013514776576}]
I20260809 02:19:06.663892 2110 sys_catalog.cc:455] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.663975 2110 sys_catalog.cc:458] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.663944 2132 catalog_manager.cc:1347] Generated new cluster ID: 7c17e65f97014e998b10f54d2e2d8cd4
I20260809 02:19:06.664022 2132 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260809 02:19:06.670778 2011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.660239 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34178 (local address 127.9.40.125:42133)
0809 02:19:06.660283 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.660284 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.660475 (+ 191us) server_negotiation.cc:407] Connection header received
0809 02:19:06.660669 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.660670 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.660683 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.660717 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.662070 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.662835 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.665261 (+ 2426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.665587 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.665637 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.667459 (+ 1822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.667460 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.667461 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.667469 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.669406 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.669819 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.669820 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.669820 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.669881 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.670101 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.670102 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.670102 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.670297 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.670376 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.670603 (+ 227us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.670686 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260809 02:19:06.671561 2137 mvcc.cc:205] Tried to move back new op lower bound from 7316447013514776576 to 7316447013496369152. Current Snapshot: MvccSnapshot[applied={T|T < 7316447013514776576}]
I20260809 02:19:06.679031 2006 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } attempt: 3
I20260809 02:19:06.681181 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.660504 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58642 (local address 127.9.40.124:39793)
0809 02:19:06.660545 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.660547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.668879 (+ 8332us) server_negotiation.cc:407] Connection header received
0809 02:19:06.669129 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.669131 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.669143 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.669190 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.670611 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.671346 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.672878 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.673162 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.673205 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.675743 (+ 2538us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.675745 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.675745 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.675753 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.677587 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.678816 (+ 1229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.678817 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.678818 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.678881 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.680328 (+ 1447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.680330 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.680330 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.680579 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.680704 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.680966 (+ 262us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.681071 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Aug 09 02:19:06 dist-test-slave-qg60 krb5kdc[1828](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786241946, etypes {rep=17 tkt=17 ses=17}, kudu/127.9.40.125@KRBTEST.COM for kudu/127.9.40.124@KRBTEST.COM
I20260809 02:19:06.689426 1921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.660053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60240 (local address 127.9.40.126:35217)
0809 02:19:06.660100 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.660103 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.669017 (+ 8914us) server_negotiation.cc:407] Connection header received
0809 02:19:06.669774 (+ 757us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.669775 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.669788 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.669819 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.672856 (+ 3037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.673520 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.674552 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.674863 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.674932 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.678209 (+ 3277us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.678210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.678210 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.678218 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.681853 (+ 3635us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.683749 (+ 1896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.683750 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.683751 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.683843 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.686965 (+ 3122us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.686967 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.686967 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.687126 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.687234 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.689203 (+ 1969us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.689313 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:06.689821 2132 catalog_manager.cc:1370] Generated new certificate authority record
I20260809 02:19:06.690442 2010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.679652 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.9.40.124:39793 (local address 127.0.0.1:58650)
0809 02:19:06.679706 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0809 02:19:06.679717 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0809 02:19:06.679773 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0809 02:19:06.681480 (+ 1707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0809 02:19:06.681481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0809 02:19:06.681651 (+ 170us) client_negotiation.cc:189] Negotiated authn=SASL
0809 02:19:06.681709 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.681711 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.682847 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.682849 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.683618 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0809 02:19:06.683621 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.684374 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.684375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0809 02:19:06.684446 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.684969 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0809 02:19:06.684974 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0809 02:19:06.686511 (+ 1537us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0809 02:19:06.689072 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.689073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.689073 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.689307 (+ 234us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.689731 (+ 424us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0809 02:19:06.689732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0809 02:19:06.689732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0809 02:19:06.689801 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0809 02:19:06.690120 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0809 02:19:06.690121 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0809 02:19:06.690169 (+ 48us) client_negotiation.cc:769] Sending connection context
0809 02:19:06.690236 (+ 67us) client_negotiation.cc:240] Negotiation successful
0809 02:19:06.690343 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
I20260809 02:19:06.690745 2093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:06.679911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58650 (local address 127.9.40.124:39793)
0809 02:19:06.681262 (+ 1351us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.681264 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.681273 (+ 9us) server_negotiation.cc:407] Connection header received
0809 02:19:06.681295 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.681295 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.681307 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.681366 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.682062 (+ 696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.682780 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.683944 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:06.684250 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:06.684308 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:06.686619 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0809 02:19:06.686620 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0809 02:19:06.686621 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0809 02:19:06.686628 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0809 02:19:06.688990 (+ 2362us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.689588 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.689590 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.689590 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.689650 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0809 02:19:06.689872 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0809 02:19:06.689873 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0809 02:19:06.689873 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0809 02:19:06.690025 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0809 02:19:06.690088 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:06.690573 (+ 485us) server_negotiation.cc:299] Negotiation successful
0809 02:19:06.690662 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1334,"thread_start_us":163,"threads_started":1}
I20260809 02:19:06.691185 2132 catalog_manager.cc:1504] Loading token signing keys...
I20260809 02:19:06.692762 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: Bootstrap starting.
I20260809 02:19:06.693663 2006 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: Neither blocks nor log segments found. Creating new log.
I20260809 02:19:06.694007 2006 log.cc:824] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: Log is configured to *not* fsync() on all Append() calls
I20260809 02:19:06.700620 2132 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f: Generated new TSK 0
I20260809 02:19:06.701031 2132 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260809 02:19:06.708189 2006 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: No bootstrap required, opened a new log
I20260809 02:19:06.710381 2006 raft_consensus.cc:348] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.710578 2006 raft_consensus.cc:729] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 22376c134461482a9e998f27060e8fa8, State: Initialized, Role: FOLLOWER
I20260809 02:19:06.710660 2006 consensus_queue.cc:260] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } }
I20260809 02:19:06.711002 2006 sys_catalog.cc:564] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: configured and running, proceeding with master startup.
I20260809 02:19:06.711557 1975 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "01e4b84dc4654708b711f6819346497f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "22376c134461482a9e998f27060e8fa8"
I20260809 02:19:06.711683 1975 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [term 0 FOLLOWER]: Advancing to term 1
I20260809 02:19:06.712617 1975 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 01e4b84dc4654708b711f6819346497f in term 1.
I20260809 02:19:06.713042 1973 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [term 1 FOLLOWER]: Refusing update from remote peer 01e4b84dc4654708b711f6819346497f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260809 02:19:06.713447 2096 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [LEADER]: Connected to new peer: Peer: permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260809 02:19:06.716914 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.717016 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.717067 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 01e4b84dc4654708b711f6819346497f. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.717114 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: This master's current role is: FOLLOWER
I20260809 02:19:06.717142 2143 sys_catalog.cc:455] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:06.717187 2143 sys_catalog.cc:458] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8 [sys.catalog]: This master's current role is: FOLLOWER
W20260809 02:19:06.718432 2157 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260809 02:19:06.718494 2157 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260809 02:19:06.719074 2159 mvcc.cc:205] Tried to move back new op lower bound from 7316447013680504832 to 7316447013496369152. Current Snapshot: MvccSnapshot[applied={T|T < 7316447013514776576}]
W20260809 02:19:06.734999 2093 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.733377 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58658 (local address 127.9.40.124:39793)
0809 02:19:06.733435 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.733437 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.734185 (+ 748us) server_negotiation.cc:407] Connection header received
0809 02:19:06.734300 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.734302 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.734313 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.734346 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.734902 (+ 556us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58658: BlockingRecv error: recv got EOF from 127.0.0.1:58658 (error 108)
Metrics: {"server-negotiator.queue_time_us":30}
W20260809 02:19:06.736018 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.733854 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34184 (local address 127.9.40.125:42133)
0809 02:19:06.733898 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.733900 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.735289 (+ 1389us) server_negotiation.cc:407] Connection header received
0809 02:19:06.735406 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.735408 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.735421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.735457 (+ 36us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.735950 (+ 493us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34184: BlockingRecv error: recv got EOF from 127.0.0.1:34184 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:19:06.737329 1921 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.736427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60244 (local address 127.9.40.126:35217)
0809 02:19:06.736477 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.736479 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.736592 (+ 113us) server_negotiation.cc:407] Connection header received
0809 02:19:06.736702 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.736703 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.736714 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.736765 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.737261 (+ 496us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60244: BlockingRecv error: recv got EOF from 127.0.0.1:60244 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260809 02:19:06.739899 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.739071 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34188 (local address 127.9.40.125:42133)
0809 02:19:06.739112 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.739114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.739322 (+ 208us) server_negotiation.cc:407] Connection header received
0809 02:19:06.739432 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.739433 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.739452 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.739494 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.739827 (+ 333us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34188: BlockingRecv error: recv got EOF from 127.0.0.1:34188 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:19:06.743219 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.742373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34204 (local address 127.9.40.125:42133)
0809 02:19:06.742409 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.742411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.742694 (+ 283us) server_negotiation.cc:407] Connection header received
0809 02:19:06.742795 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.742796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.742807 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.742836 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.743165 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34204: BlockingRecv error: recv got EOF from 127.0.0.1:34204 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 02:19:06.747265 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.746369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34220 (local address 127.9.40.125:42133)
0809 02:19:06.746419 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.746421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.746710 (+ 289us) server_negotiation.cc:407] Connection header received
0809 02:19:06.746817 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.746818 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.746830 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.746861 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.747210 (+ 349us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34220: BlockingRecv error: recv got EOF from 127.0.0.1:34220 (error 108)
Metrics: {"server-negotiator.queue_time_us":18}
W20260809 02:19:06.761651 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.757122 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34236 (local address 127.9.40.125:42133)
0809 02:19:06.757173 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.757175 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.760990 (+ 3815us) server_negotiation.cc:407] Connection header received
0809 02:19:06.761117 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.761118 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.761132 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.761164 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.761583 (+ 419us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34236: BlockingRecv error: recv got EOF from 127.0.0.1:34236 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
W20260809 02:19:06.772092 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.771110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34246 (local address 127.9.40.125:42133)
0809 02:19:06.771157 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.771159 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.771456 (+ 297us) server_negotiation.cc:407] Connection header received
0809 02:19:06.771564 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.771565 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.771577 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.771608 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.772017 (+ 409us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34246: BlockingRecv error: recv got EOF from 127.0.0.1:34246 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260809 02:19:06.790707 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.789671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34252 (local address 127.9.40.125:42133)
0809 02:19:06.789717 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.789720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.790059 (+ 339us) server_negotiation.cc:407] Connection header received
0809 02:19:06.790173 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.790174 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.790188 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.790222 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.790644 (+ 422us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34252: BlockingRecv error: recv got EOF from 127.0.0.1:34252 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:19:06.825163 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.824201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34268 (local address 127.9.40.125:42133)
0809 02:19:06.824248 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.824250 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.824532 (+ 282us) server_negotiation.cc:407] Connection header received
0809 02:19:06.824633 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.824634 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.824646 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.824676 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.825103 (+ 427us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34268: BlockingRecv error: recv got EOF from 127.0.0.1:34268 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
W20260809 02:19:06.891580 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:06.890776 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34282 (local address 127.9.40.125:42133)
0809 02:19:06.890825 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:06.890827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:06.890968 (+ 141us) server_negotiation.cc:407] Connection header received
0809 02:19:06.891067 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:06.891068 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:06.891079 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:06.891160 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:06.891512 (+ 352us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34282: BlockingRecv error: recv got EOF from 127.0.0.1:34282 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260809 02:19:07.021837 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:07.020956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34288 (local address 127.9.40.125:42133)
0809 02:19:07.021092 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.021095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.021201 (+ 106us) server_negotiation.cc:407] Connection header received
0809 02:19:07.021300 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.021301 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.021313 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.021372 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:07.021773 (+ 401us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34288: BlockingRecv error: recv got EOF from 127.0.0.1:34288 (error 108)
Metrics: {"server-negotiator.queue_time_us":103}
I20260809 02:19:07.172915 2096 sys_catalog.cc:455] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "01e4b84dc4654708b711f6819346497f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "01e4b84dc4654708b711f6819346497f" member_type: VOTER last_known_addr { host: "127.9.40.126" port: 35217 } } peers { permanent_uuid: "22376c134461482a9e998f27060e8fa8" member_type: VOTER last_known_addr { host: "127.9.40.125" port: 42133 } } peers { permanent_uuid: "80a2f2bfa8ee43f1be816214617e2ace" member_type: VOTER last_known_addr { host: "127.9.40.124" port: 39793 } } }
I20260809 02:19:07.173080 2096 sys_catalog.cc:458] T 00000000000000000000000000000000 P 01e4b84dc4654708b711f6819346497f [sys.catalog]: This master's current role is: LEADER
W20260809 02:19:07.280077 2011 negotiation.cc:337] Failed RPC negotiation. Trace:
0809 02:19:07.279400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34292 (local address 127.9.40.125:42133)
0809 02:19:07.279498 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.279500 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.279517 (+ 17us) server_negotiation.cc:407] Connection header received
0809 02:19:07.279547 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.279548 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.279558 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.279633 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0809 02:19:07.279984 (+ 351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:34292: BlockingRecv error: recv got EOF from 127.0.0.1:34292 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260809 02:19:07.652848 2122 catalog_manager.cc:1259] Loaded cluster ID: 7c17e65f97014e998b10f54d2e2d8cd4
I20260809 02:19:07.652925 2122 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: loading cluster ID for follower catalog manager: success
I20260809 02:19:07.653731 2122 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: acquiring CA information for follower catalog manager: success
I20260809 02:19:07.653990 2122 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 80a2f2bfa8ee43f1be816214617e2ace: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:19:07.718945 2157 catalog_manager.cc:1259] Loaded cluster ID: 7c17e65f97014e998b10f54d2e2d8cd4
I20260809 02:19:07.719025 2157 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: loading cluster ID for follower catalog manager: success
I20260809 02:19:07.719790 2157 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: acquiring CA information for follower catalog manager: success
I20260809 02:19:07.720037 2157 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 22376c134461482a9e998f27060e8fa8: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260809 02:19:07.796274 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:07.793489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34304 (local address 127.9.40.125:42133)
0809 02:19:07.793650 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.793652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.793666 (+ 14us) server_negotiation.cc:407] Connection header received
0809 02:19:07.793742 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.793743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.793797 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.793871 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:07.794057 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.794572 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.795392 (+ 820us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.795594 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.795653 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:07.795855 (+ 202us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:07.795976 (+ 121us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:07.796050 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:07.796119 (+ 69us) server_negotiation.cc:299] Negotiation successful
0809 02:19:07.796139 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":89,"threads_started":1}
I20260809 02:19:07.800905 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:07.798274 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58668 (local address 127.9.40.124:39793)
0809 02:19:07.798419 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.798421 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.798432 (+ 11us) server_negotiation.cc:407] Connection header received
0809 02:19:07.798483 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.798484 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.798536 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.798661 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:07.798797 (+ 136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.799289 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.800029 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.800247 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.800325 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:07.800494 (+ 169us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:07.800619 (+ 125us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:07.800712 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:07.800775 (+ 63us) server_negotiation.cc:299] Negotiation successful
0809 02:19:07.800799 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
I20260809 02:19:07.805687 2241 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:07.802687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58682 (local address 127.9.40.124:39793)
0809 02:19:07.802732 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.802734 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.802929 (+ 195us) server_negotiation.cc:407] Connection header received
0809 02:19:07.803031 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.803033 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.803045 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.803075 (+ 30us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:07.803265 (+ 190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.803763 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.804868 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.805173 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.805242 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:07.805374 (+ 132us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:07.805406 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:07.805454 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:07.805573 (+ 119us) server_negotiation.cc:299] Negotiation successful
0809 02:19:07.805603 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260809 02:19:07.805977 2248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:07.802668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60260 (local address 127.9.40.126:35217)
0809 02:19:07.802790 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.802792 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.802918 (+ 126us) server_negotiation.cc:407] Connection header received
0809 02:19:07.803051 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.803052 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.803110 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.803193 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:07.803367 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.804137 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.805020 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.805344 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.805402 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:07.805559 (+ 157us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:07.805683 (+ 124us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:07.805748 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:07.805861 (+ 113us) server_negotiation.cc:299] Negotiation successful
0809 02:19:07.805890 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98,"thread_start_us":72,"threads_started":1}
I20260809 02:19:07.806087 2235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0809 02:19:07.802789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34310 (local address 127.9.40.125:42133)
0809 02:19:07.802844 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0809 02:19:07.802847 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0809 02:19:07.802910 (+ 63us) server_negotiation.cc:407] Connection header received
0809 02:19:07.802990 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0809 02:19:07.802992 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0809 02:19:07.803006 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0809 02:19:07.803064 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0809 02:19:07.803207 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.804101 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.805203 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0809 02:19:07.805547 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0809 02:19:07.805609 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0809 02:19:07.805786 (+ 177us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0809 02:19:07.805821 (+ 35us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0809 02:19:07.805864 (+ 43us) server_negotiation.cc:1035] Waiting for connection context
0809 02:19:07.805969 (+ 105us) server_negotiation.cc:299] Negotiation successful
0809 02:19:07.805995 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260809 02:19:07.808125 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1844
I20260809 02:19:07.818703 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 1928
I20260809 02:19:07.825394 9377 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskQln13t/build/release/bin/kudu with pid 2014
2026-08-09T02:19:07Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2934 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5881 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (278098 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST